New Primal Pages for Publishers leverages advanced semantic technology to make Web publishing and community engagement easier than ever

DEMO Fall 2010 – SANTA CLARA, CA, September 14, 2010 — Live today from the 2010 DEMO Fall Conference, Primal, a pioneer of Internet automation technology, officially launched an exciting new product.  Primal Pages for Publishers provides a simpler, more convenient way to assemble and publish Web content. With just a few clicks, Pages for Publishers automatically generates unique web pages based on the interests of individual people in the publisher’s audience. Following the launch of its semantic technology platform and thought networking products for consumers in June, Primal presents the company’s latest innovation at the world-renowned DEMO conference, taking place this week at the Silicon Valley Hyatt Regency in Santa Clara September 13 – 15, 2010.

Primal Pages for Publishers automates the entire publishing process, from identifying topics to provisioning content to organizing information. It also provides a dynamic and vibrant medium for engaging readers directly in the process. This not only saves publishers valuable time and money but also providers their audience with a more personalized and engaging online experience. For innovative publishers large and small, Primal Pages delivers increased revenue opportunities, more reader engagement, and reduced costs via a scalable, customizable platform.

“We recognized that individuals and organizations were really struggling to create online content, so we decided to provide people with a dead-simple way to get that job done,” said Primal’s Founder and Co-President Peter Sweeney. “We’re streamlining the management of even the broadest subject matter and tailoring the user experience to markets of one. With Primal Pages, the audience effortlessly personalizes their online experience, under the editorial guidance of the publisher, leading to a much richer experience for all.”

Pages for Publishers enables media outlets, independent publishers, and individuals to augment their online presence with fresh content from around the Web, driven by the specific interests of each individual reader. Pages can incorporate a range of content sources based on the requirements of the publisher, such as blogs, book recommendations, images, news and other reference material.

Primal technology allows their readers to imagine and architect content in real-time rather than placing the burden on the publishers to build it in advance. Further, by drawing upon information from the entire Web, Pages for Publishers offers the publisher’s audience a compelling reason to stay engaged.  By providing a scalable, customizable platform, Primal dramatically reduces the costs of maintaining large websites and communities. Primal plans to offer ad-supported and transactional pricing models to support publishers of all sizes.

“Consumers are looking for instant gratification.  Primal now offers Web publishers a way to leverage the mass amount of content on the Web to provide a seamless, complete experience in one simple view.  We feel that this implementation of semantic technology will be a game changer for the industry,” said Matt Marshall, executive producer of DEMO.

Primal Pages is different from traditional approaches that utilize teams of people to search and seek out information on a specific topic. Primal applies a unique and patented semantic synthesis technology to completely automate the process, saving publishers significant time and money. Today, valuable pieces of information are scattered across the Internet requiring publishers to troll the Web, visiting various destinations to find the information they need. Now, with Primal Pages for Publishers, publishers can painlessly supplement their original content with dynamic, relevant, and valuable content from across the Internet.

Peter Sweeney, along with Robert Barlow-Busch, will be presenting Primal’s Pages for Publishers live from the DEMO stage on Wednesday, September 15 and will be located at Station #13 in the Pavilion.

Technology was supposed to revolutionize our lives. There were promises of 20-hour work weeks, robotic servants to do our bidding, and leisurely weekday afternoons in the sun. That was a fantastic dream. So what happened along the way?

Today, we face the grim reality that most of the technology we build simply enables people to do more work.

Your PC is perhaps the best example of this. Sure, it’s a powerful tool. But it’s one that can do almost nothing without a human driving it. You respond to your emails. You browse the Web. You write that report. And you fix it when it breaks.

Could a computer do some of that work for you?

The promise of automation

Automation is the missing link that’s needed to make technology work for us. When a technology automates something, it completes a task that’s valuable to us, but in a manner that doesn’t require our personal oversight.

There’s evidence of automation in our lives today. We have simple assistants that can finish tasks like “wash and dry these dishes” (your dishwasher) or “vacuum this carpet” (a Roomba). Even your PC, problem-generating nightmare that it is, can handle a command to “find and correct the obvious typos in this document” (a spell checker).

But “write my science paper for me” is not a task your computer is prepared to handle. Neither is “buy an attractive couch for my living room” or “think of a good topic for my next blog post.”

Combining automation with the Web

The web is a lot of work.Image source: Flickr, http://www.flickr.com/photos/epac_island/391204177/

To accomplish these kinds of tasks, most people use the Web.

Unfortunately, the Web today is a lot of work. It’s full of repetitive chores like entering keyword variations, experimenting with different services, paging through search results, and patching together information from many sources.

To accomplish a simple task, you might find yourself with 10 browser tabs open, each with a mile-long history of different services and search terms you’ve explored. Quite likely, you’ve also patched together an email to your colleagues, with notes you’ve hastily copied and pasted.

Using semantic technology to enable new kinds of automation

Before a computer assistant could write a document or go shopping for you, you would need a way to communicate your requirements to it. Your assistant needs a sense of the boundaries it has to work within.

Semantic technology provides a way to draw those boundaries. It provides a formal way for humans to tell computers what it is they need. Without semantics, a computer assistant tasked with writing a paper about “Tesla” couldn’t perceive the topic as anything more than a string of letters. It could do nothing more than patch together every paragraph it finds containing the word “Tesla”.

With semantic technology, “Tesla” becomes a concept rich with meaning. Suddenly, your assistant can understand that you were referring to Tesla the scientist and not Tesla Motors. Even more, if this knowledge model represents your specific ideas and perspectives, as opposed to some universal worldview, your assistant could perform highly personalized work.

With this knowledge model, your computer assistant can perform more complex automation. In fact, it could create a whole website for you about your personal perspective on Tesla.

After all, shopping assistants aren’t much helped by knowing couches are pieces of furniture. To fulfill the promise of true automation, they need to be driven by your particular thoughts and intentions.  The shopping assistant needs to know where you plan to put the couch, why you want it, and what your budget is.

Conclusion: The Web as your personal assistant

The Web experience of the future will feel more like interacting with a personal assistant and less like doing a bunch of chores. By providing software agents that understand your thoughts and intentions, the Web will begin to relieve your burdens rather than add to them.

Maybe there’s still a fighting chance for those leisurely weekday afternoons in the sun.

To help illustrate Primal’s vision for what the future holds, we’ve decided to tell a story about thought networking: comic-style! Click below to read the comic, or download a PDF version to read offline.

By Peter Sweeney (@petersweeney)

Beginning last year, Michael Arrington and many others identified content farms as the great disruptor, an innovation that will eventually eclipse traditional and social media. With their vast pools of cheap distributed labor, farms would glut the market with low quality content, overwhelming older approaches.

Demand Media is the lightning rod for content farming. But with the information disclosed in their recent IPO filing, it’s clear that content farms are not the great disruptive innovation that people fear.

Figure 1: Demand Media is the lightning rod for content farming
(Image source: Demand Media)

To declare an innovation truly disruptive, it needs to embody some rare qualities. High on the list are productivity and cost advantages. Disruptive innovations bring massive improvements in productivity. The cost advantages allow the innovation to permeate to entirely new markets and usage scenarios, profoundly shaking the incumbents and delighting new customers.

Content farms don’t exhibit these productivity advantages. Demand Media, purportedly among the most sophisticated farms in terms of their operational processes, are reporting underwhelming results. While farms are automating aspects of the content manufacturing process, they remain labor intensive, hand rolled operations. When compared against older models, farms are not demonstrating the productivity advantages that would be needed to qualify as a disruptive innovation.

Figure 2: Content farms employ large numbers of writers, often freelancers, to create content
(Image source: Flickr, http://www.flickr.com/photos/bcostin/366629702)

Another crucial aspect of disruptive innovations is the element of surprise. Disruptive innovations are disruptive because incumbents often deny or ignore them until it’s too late. These innovations challenge our fundamental assumptions about what’s possible. As such, it’s difficult for those outside the innovators’ circle to appreciate the impact.

Consider social media, exhibited in projects like Wikipedia, as truly disruptive. When Wikipedia began, even those close to the project didn’t think it was possible to build a world class encyclopedia through a massive collaboration of strangers and amateurs. It was only through these types of grand experiments that the disruptive implications of Web 2.0 and social media became widely understood.

Again, content farms fail to exhibit this quality of disruptive innovation. Even in the earliest days, few were denying that content farming is possible. When you parse the attributes of content farms, they are understood by all as incremental innovations. Their main differentiation is in the commercial aspects: applying distributed and collaborative approaches to content manufacturing is old news; marrying those aspects to a sophisticated commercial enterprise is the incremental innovation.

Demand Media may disagree. In their SEC filing, Demand describes their differentiation as follows:

While traditional media companies create content based on anticipated consumer interest, we create content that responds to actual consumer demand. Our approach is driven by consumers’ desire to search for and discover increasingly specific information across the Internet. By listening to consumers, we are able to create and deliver accurate and precise content that fulfills their needs.

A fine purpose, but again, on closer examination, it’s hardly disruptive. The difference between anticipated consumer interest and actual consumer demand even sounds incremental, graduating along the same axis of consumer demand. There’s certainly no aspect of fundamentally challenging our assumptions of what’s possible. Publishers have been eking out incremental improvements in market segmentation forever. Even the specific approaches of Demand Media are exhibited across the activities of domainers and arbitrageurs, even reverse engineered in the patents of the most powerful search company in the world.

Content farms like Demand Media are important. They’ve demonstrated the scale under which social media projects can be conducted as commercial publishing ventures. They’re important, but in a very incremental way. Unlike truly disruptive innovations, they won’t put content industries on a new growth trajectory.

None of this is to suggest that the content industries should rest easy. There’s certainly a fundamental disruption in content on the horizon, but you won’t find it on the farm. Rather, the magnitude of productivity advantages and assumption-challenging capabilities needed are found only in factories, in far more industrialized and automated approaches. And although most will deny or ignore their presence until it’s too late, content factories will make farming seem quaint in comparison.

We’ll look at content factories in more detail in a future post. Follow us on Twitter or subscribe to our RSS feed.

Figure 3: Content factories create machines that automate content manufacturing
(Image source: Primal, http://about.primal.com/2009/11/are-web-factories-stealing-your-job/)

News flash: Content farms and factories continue their explosive growth, while traditional publishers and newspapers free fall.

Machines and godless industrialists are running amok, polluting the Web with low rent content. Oh the humanity! Where will we find a little human redemption in this scourge of machine generated content?

Why, “News For You!” say some industrialists. Farms and factories may find redemption by adding social and personalization enhancements. Recently Google News, that granddaddy of content factories, unveiled its most significant overhaul since its launch in 2002. Features such as a personal news stream, where consumers can edit categories and content sources to suit their needs, bring that human touch to machine automation.

Google News: News for you

“We’ll save you!” say some experts. Farms and factories may redeem themselves through partnerships with legitimate organizations. Begrudgingly, traditional news organizations are partnering with farms like Demand Media, AOL Seed, and the Examiner.com. Even Google News is bringing in a select group of human editors as partners.

While all interesting developments, this reporter remains unconvinced. Such discussions and industry moves are suspiciously old school, overwhelmingly focused on the quality of content produced. Content farms and factories are often written off as sub-human attempts at journalism. It echoes earlier criticisms of blogging and citizen journalism.

Then as now, market disruptions cannot be evaluated against old standards. Content farms and factories need a new value assessment, one that has nothing to do with journalistic quality (or the lack thereof). Quality has already grossly overshot the expectations of many consumers, which is precisely why the industry is so ripe for disruption. And however glamorous it may seem to incumbents, most consumers don’t want to edit and curate the news any more than they want to report and write it. This self-absorbed supply-side perspective is leaving traditional journalists ill equipped to adjust to these changes.

Maybe there is a deeper, more pervasive humanity supporting the rise of content farms and factories. Consumers want content that is supportive of their immediate needs. They want convenience and simplicity, to be treated as markets of one. Content farms and factories succeed in this fine grained segmentation. Any perceived shortcomings in quality are more than overcome by the convenience benefits: content that is more individually task-oriented and crafted to the readers’ unique requirements.

“Create by consuming”, says Primal. Last week, our factory took another step towards our vision of consumer-directed content manufacturing, with the release of Primal Pages. We don’t expect consumers to act like journalists, but we do want to empower them as the ultimate architects of the content they consume. Under our model, consumers direct the automated content manufacturing in real-time — as they’re consuming it — through the topics they specify and their unique click-paths.

Primal Pages: Create content by browsing

By journalistic standards, this may be publishing at its most basic. But that’s the entire point: a drive towards ultimate simplicity and convenience for consumers as consumers.

The sad truth is that most incumbents are failing to examine the true humanity in content farms and factories in the motivations of the consumers. Eric Schmidt, describing the “cultural phenomenon” that is Google, captures it succinctly: “It’s a way in which consumers consume information.”

First and foremost, consumers consume information, preferring the most tailored and convenient information possible.

Posted by Peter Sweeney

The current friction between people who create content and those who create content factories evokes the evolution versus intelligent design debate. Why do the intelligent designers of content feel under siege as content factories create ever more gluttonous methods for manufacturing it? What does each side have to teach the other? Here are the arguments.

The intelligent design hypothesis

Professional content creators are adopting the posture of intelligent designers. They’re unquestionably intelligent; no one knows more about high quality content and how to create it. Here’s their perspective:

The skills involved in creating artful prose or insightful journalism has an irreducible complexity. Machines simply cannot create quality content. Content that is manufactured by machines is, at best, an unintelligently cobbled together hodgepodge of regurgitated information.

The reason content factories are so economically successful is mass production. They compete not on quality, but on quantity. Search engines and aggregators are complicit in this abuse of the Web, force feeding a glut of low quality content to unsuspecting consumers.

Professional content creators are under siege because content factories are stealing our jobs and glutting the market with low quality content. The market for hand crafted content is eroding as the value of content depreciates, making it economically impossible for professionals to practice our trade.

Inevitably, our hand crafted content could be wiped out of existence, and all of us will suffer for it. Everyone should be concerned.

The evolution hypothesis

Content factories are adopting the posture of evolutionists: provide for the basic elements of an evolutionary process—reproduction, mutation, and selection—and allow the content to evolve and adapt to meet the needs of consumers. Here’s their perspective:

Quality is subjective; good enough is good enough. Consumers care mostly about the time and effort needed to complete their tasks. They have an unapologetically demand-side bias. Consumers would rather have low-quality content that is more specific to their immediate tasks than high-quality content directed to a more general audience.

Content factories are successful due to our adaptive approach. We encode memes in content. Tiny mutations in these memes may seem grotesque on the surface, but in the aggregate they provide consumers with many variations on a theme. From this diversity, search engines allow consumers to make their own selections, which are fed back to the content factories to inform the next iteration.

Professional content creators are under siege only because they are slow to adapt. Their value assessments of the content are self-serving, out of step with their markets.

But it’s not the end of the world for content creators. While formulaic content will continue to move from human creators to machines, markets for high quality content will survive this industrial revolution, just as hand crafted goods survived mass production. Those that adapt will thrive, those that don’t will be marginalized.

Evolution or intelligent design?

Content professionals need to face the realities of Web 3.0 industrialization, just as they had to confront the social revolution of Web 2.0. While technological change is disruptive, it also brings new opportunities.

For their part, the content factories face the same competitive pressures as any other industry. No amount of productivity will protect those that fail to find markets that value their content. And these competitive pressures will also force factories to continue to ascend the quality ladder, often by incorporating the knowledge and best practices of content professionals.

Unlike the biological world, on the Web, evolution and intelligent design are reconcilable.

Industrialization is transforming our information economy, destroying old business models and creating new opportunities. The impact it will have on new media will make Web 2.0 seem tame in comparison. To understand this transformation and leverage it effectively, you need to parse the myths from reality.

Back in May 2009, I argued that industrialization is the most transformative force on the Internet. Human tasks are rapidly being displaced by machines. Factories of advanced technologies are being constructed to automate the manufacture of information and content. Predictably, there is much hand-wringing and righteous indignation expressed about this economic sea change.

Primal Fusion just returned from the Defrag conference where industrialization was a focus of the conversations. Andy Kessler’s keynote discussed how industrialization will destroy jobs. It was lambasted as dystopian and ridiculous. Stowe Boyd spoke of social media leading to a post-industrial world; my talk (slides, coverage) discussed the present reality of Internet industrialization.

A recent Wired article on Demand Media was another kick to the industrial hive. This “fiendishly clever” company is automating aspects of the editorial process and manufacturing workflow, creating prodigious amounts of content to fill the nooks and crannies of the online world. Predictably, the response has been overwhelmingly negative.

The concerns are real. Industrialism affects lives. Jobs are being lost and venerable institutions are being eradicated. Unfortunately, gauging by the reaction in the blogosphere, and through my own travels venturing in this area, people are awash in misconceptions and half-truths about this disruptive force.

Myth #1: Industrialism is about automating jobs

The criticisms of industrialism can largely be boiled down to one main indictment: factories destroy jobs. They marginalize skilled professionals and subjugate their roles.

While there is some truth in this criticism, it is also superficial and misleading. There is a naïve view of a “sweat shop” migration, where formerly highly paid professionals end up toiling away at a pauper’s wage creating the drivel that editorial machines assign to them.

The reality is far more nuanced. The technologists who enable this transformation are not automating whole jobs or professions, but rather discrete tasks.

For example, consider the process of manufacturing online content. It’s an extremely complicated process that involves a large number of highly specialized tasks: workflow management, technical infrastructure, editorial and creative direction, content aggregation, layout, information architecture, and many others. As these tasks are automated, fewer people need to be involved in the process. Costs decrease and new economies of scale are realized.

Productivity increases with automation

Forward-looking professionals adapt to technology opportunistically. They evolve new roles that incorporate these tools.

Myth #2: Internet industrialism is about the future

Some take comfort in automation as science fiction, far removed from our current economy.

In reality, the drive towards these productivity advantages is relentless and in full force, right now. Billions are flowing into industrialized Internet industries. (note 1) Giants like Google and Amazon have already emerged. A large number of well-financed and industrious start-ups are rushing in to fill needs for specialized machinery. All are leveraging the network of the open Internet to drive this industrial revolution forward.

Denying this present economic reality, or imagining that concurrent social revolutions will displace this economic drive towards productivity, would be a risky response.

Internet Industrial Revolution

Myth #3: Internet factories create spam

Content professionals take pride in the quality of their hand-crafted content. They imagine machines will only create spam, polluting the Internet with a glut of low-quality content that consumers will reject in droves.

It’s painful to watch critics of industrialism pulling out their quality knives in this quantity gun fight. While artisans decry the death of professional content, the factories quietly find markets where the quality of their content is valued as “good enough”.

There are no absolute measures of quality. It needs to be assessed in the context of our insatiable demand for information. People value content that makes their tasks simpler and more convenient; content that is evermore customized to their needs. Assessments of quality need to reconcile these fast-changing dynamics of supply and demand.

Is there room for abuse? Of course; we will struggle with content spam just as surely as we struggle with spam in all its existing forms. But quality, as an argument against industrial productivity, is a misdirection you’ll want to avoid. Just ask the publishing industry.

Are Web factories stealing your job?

The answer is a resounding no. Jobs and professions are more than collections of tasks. But factory builders are gunning for formulaic and repetitive tasks that lend themselves to automation. If you wear tasks like comfortable old shoes, your job will be overrun. If you embrace these technologies as just another tool in your professional toolbox, you’ll run these factories.

If you’d like to check out some examples of Web factories, the slides from my Defrag presentation are online. If you want to get involved with Primal Fusion in building one of these factories, check out our alpha and careers.

Notes

1. Semantic technologies are a key enabler for automation. Project10X
   projects that the market for semantic technologies worldwide will exceed
   US$50-billion in 2010.

In a recent article in The Guardian, Cory Doctorow called for search reform. “Search is the beginning and the end of the internet,” he wrote. While I agree things need to change, a reformed Internet built around search is like a reformed energy policy built around oil. If we only look for solutions through the narrow lens of search, we’re unlikely to solve these problems at all.

Cory’s article is a tight encapsulation of widely held concerns over search. We share too much private information with search engines. They hold too much power over what is relevant and important. And there is a troubling lack of transparency and accountability into how search engines weigh these decisions.

His bottom line: Vesting this kind of power with a handful of companies is a “terrible idea”.

Fair enough, but when he ventures into possible solutions, Cory, like so many other Internet watchers, makes a serious wrong turn. “Put that way, it’s obvious: if search engines set the public agenda, they should be public.”

He’s correct in pointing out that we need a “transparent, participatory solution”. But he’s taxing search as both the villain and the liberator, when neither label holds.

This is the crux of a seemingly universal misconception: that search will always be the beginning and the end of the Internet. It seems impossible for many to imagine a next-generation Internet that doesn’t have searching at its core.

But dynasties rise and fall, and so will search as our primary online experience.

It’s happening now. There are complementary approaches to validate that search is not the beginning and the end of the Internet. Social networking is the most obvious example. Many people now prefer to tap their social network rather than searching for the information themselves.

Looking further out, companies like Primal Fusion and Siri are advocating for new consumer interaction models based in computer agency. Rather than using computers as tools, we can task them like personal assistants, without involving people at all.

The answer to the problems cited above isn’t in reinventing open versions of successful commercial initiatives, but rather relentless innovation forward. We need to broaden the online environment with new networks and new consumer interaction models.

More concretely, Google knows nothing of my search when it’s brokered through my social networks and computer assistants. A distributed Internet experience mitigates the risks of centralized services.

So the next time you hear a suggestion for a better, faster, smarter search, repeat after me: Search, baby, search! Then go imagine a bigger, broader, richer way to experience the Internet.

Primal Fusion is pleased to support this discussion by sponsoring a lecture by Cory Doctorow, Copyright versus Universal Access to All Human Knowledge and Groups Without Cost. It will take place on Thursday, October 22 at 4:00 PM at the Perimeter Institute in Waterloo, Ontario, Canada, as part of Quantum to Cosmos (Q2C): Ideas for the Future. We hope to hear from you, at the event and online.

How Do We Roll Out The Semantic Web? Paradoxically, the fast track may involve getting help from billions of people who know nothing about the Semantic Web and have no interest in it.

Challenges with current approaches

Most of the current approaches to building the Semantic Web focus on content. We create semantic representations of existing assets such as databases, documents, and social media. Machines “read” this knowledge and execute tasks on behalf of consumers. In the Semantic Web world, the approach is content first, consumers second.

Unfortunately, semantifying content is proving to be an extraordinarily daunting task. When we expand the scope of the problem beyond our existing content assets to include knowledge generally, in all its subjective and boundless glory, the challenges of a content-first approach becomes clear. We need alternative strategies, and more importantly, many hands on the problem. (Note 1: Problems)

Some of the content-first datasets published in the Linking Open Data community project.

The consumer-first approach

One such alternative approach reverses the order of operations. It puts the focus on consumers before content. On the surface, this may seem outlandish but, in many ways, it provides a more semantic and scalable roadmap.

Under a consumer-first approach, we build systems that let consumers create the semantics that reflect how they approach, interact with, and experience content. Rather than describing the content, we represent aspects of the consumers’ thoughts, perspectives, and intentions. These consumer knowledge models can then impose a structure on the mass of unstructured content on the Web.

The major difference here is that the semantics and structure are discovered through the expectations that consumers have for the content, rather than being imposed by knowledge engineers in advance.

Note also that consumer-first is not Web 2.0. While Web 2.0 collaborative processes are obviously consumer driven, they are often framed within the task of annotating content as opposed to annotating the mental models of the consumers themselves. Further, the complexity of semantics and knowledge representation demands a Web 3.0 industrial approach to simplify things for consumers.

Example of consumer-first

As an example, let’s take a real estate application. Under a content-first approach, developers would provide some structure to organize the real estate properties in the database. To be effective, they would attempt to anticipate the needs of the consumers, and semantically annotate the property listings accordingly. Finally, consumers are asked to search through these structured listings to find a property that meets their needs.

Within a consumer-first approach, the system would begin by creating a mental model of the consumer’s requirements, a representation of their “dream home” if you like. Once the consumer articulates their “dream home”, it is used as a lens through which the property listings could be evaluated.

Given a formal structure of the consumer’s input, the property listings can remain in a largely unstructured form. However, in the process of filtering the listings through these mental models, each property could be annotated with the semantics identified by consumers.

In other words, semantic webs evolve as a by-product of this consumer-first process, instead of a bottleneck at the start. Technically, the problem shifts from the knowledge representation of existing content to the knowledge representation of abstract thought.

Technical feasibility and testing

To the question of technical feasibility, there are many existing technologies that would support this consumer-first approach: NLP to translate natural language; Web 2.0-style workflow such as semantic wikis; ontology editors; and many others. But critically, it requires a break with the past, putting the focus of the knowledge modeling unambiguously on the consumer wants instead of the producer wares. (Note 2: Technical comparisons)

At Primal Fusion, we’re testing the productivity of this consumer-first approach through our thought networking service. You can begin exploring this approach through our current alpha release (demo video, product walk-through, free registration).

Brainstorming with the Primal Fusion service generates semantic data.

Thought networks are semantic data structures that capture the mental models that people create as they work with content to complete their tasks.

Consumers brainstorm using the Primal Fusion service to create thought networks of the ideas they want to express. A semantic synthesis technology builds data structures in response to the consumers’ interactions. Once these mental models are created, software agents interact with Web services such as search engines to accomplish tasks such as searching, filtering, harvesting, and organizing information. (Note 3: User interactions)

As software agents interact with the semantic data and the Web, formerly unstructured sources are annotated with the semantics provided by consumers. In our alpha, a text filtering process categorizes and describes the content within expansive knowledge models generated by the consumers.

The result is a significant production of semantic data with minimal effort from consumers; the software provides the heavy lifting in both the construction of the thought networks and the tasks of the software agents.

Unstructured content becomes annotated with consumer semantics.

Consumers create this valuable semantic data as a by-product of their activities, costing them little in terms of their time or the skills required. At the moment, we are working to express the output in open vocabularies and plan to publish these semantic representations as Linked Data.

Scalability

In terms of scalability, this approach may seem counterintuitive. Our content assets are very tangible, while our thoughts and intentions are very abstract and seemingly boundless.

However, this focus on the consumer reduces the unbounded nature of knowledge to two very tractable dimensions: How many consumers do you need to serve and how many interactions (or tasks) do you need to support at any given time?

On closer inspection, both aspects are quite finite, and intuitively so. It is far easier for a consumer to provide a snapshot of their knowledge directed to a specific task, than for a producer to try to anticipate all the possible perspectives on their content.

Obviously, on a global scale, the Semantic Web remains an extremely large undertaking, but consumer-first offers an important scalability benefit. The factors of production—the consumers driving this process—move in lockstep with their consumption.

As consumers begin to capture their mental models and tasks in this formal way, there is a decreasing incremental cost due to the potential for individual reuse. We can reapply aspects of the same mental models to many tasks.

Billions of semantic generators

There is so much focus on the complexity and intricacies of making our content semantic and machine-readable that a simple observation is often overlooked: All of this effort is aimed at meeting the needs of people.

Creating knowledge models of content is valuable, but only one side of the coin. We also need to create knowledge models for people as the consumers of content, to give a voice to their perspectives. But most importantly, we need to recognize that every person is a semantic generator; collectively, we are the ultimate meaning-making machines. How can we better devise solutions to put those billions of semantic generators to work on building the Semantic Web?

Notes

1. Problems with content-first: The problems of scale in a production-centric model are well documented. Creating semantic representations for existing digital content is prohibitively expensive due to the amount of online content and the compounding effect in the volume of data needed to create machine-readable semantics. Further, there is a need to incorporate personal semantics, representing the interpretations and viewpoints of the consumers of the content that further complicates the problem. Lastly, as compared with the models, standards, and protocols that preceded it, the Semantic Web has placed a much higher barrier to entry on producers, a stark change from the very accessible WWW.
2. Technical comparisons: While various technologies have an affinity with a consumer-first approach, they should not be confused as implementations of a consumer-first approach. For example, NLP may be used to transform a natural language statement into a formal, machine-readable structure. Under a content-first model, it may be used as a gateway into the exploration of existing knowledge models, while under a consumer-first model, it may be used in the creation of knowledge models. Similarly, tools such as ontology editors may be used by knowledge engineers or simplified for use by laypeople, depending on the overarching implementation strategy.
3. User interactions: One of the key challenges of this consumer-first approach in the field of knowledge representation is that tactics are needed to manage the complexity of the activity. This is a key differentiator from existing Web 2.0-style approaches, in which consumers are collaborating directly and with full knowledge of the activity. Here, we must leverage implicit and indirect means to relate consumer activity to knowledge representation.

Web 2.0 is social: many hands make light work. In stark contrast, Web 3.0 is industrial: the automation of tasks displaces human work. But trite definitions won’t prepare us for change. Whatever you call it, our information economy is in the midst of an Industrial Revolution. And if you don’t place the Web within the frame of industrial manufacturing, you won’t see the real disruptive change coming.

This story reads much like the first Industrial Revolution. Artisans and skilled tradesman used to create everything by hand. Then, through the emergence of a handful of technical innovations, came the age of mass production. It was a profound turning point in human history, affecting every aspect of daily life.

Today, most content is still created by hand, the best of it by highly skilled artisans drawing on centuries of scholarship and experience. Recently, we’ve seen significant innovations in social approaches to content creation. But Web 3.0 industrialization takes content manufacturing to an entirely different level. Instead of users manually creating content, machines automate the heavy lifting. Consumers simply push the buttons and get stuff done. Think spinning wheels versus textile mills.

We’re in the midst of this Industrial Revolution right now. Billions are being spent worldwide on semantic technologies to create the factories and specialized machinery for manufacturing content. Railways of linked data and standards are being laid to allow these factories to trade and co-operate. And the most productive information services in the world are those that leverage Web 3.0 industrial processes and technologies.

Our company, Primal Fusion, is building one of these industrial Web 3.0 services. As an entrepreneur, I’ve ventured in Web 1.0, 2.0, and now 3.0. When I started in 1996, it took a team of highly skilled artisans many weeks to create a single website. Today, Primal Fusion enables individual consumers to build personal websites not in weeks or days, but in minutes, merely by brainstorming their interests. Accordingly, we’re measuring orders-of-magnitude productivity gains over Web 2.0 user-generated content models. (Note 1: Measurement)

This time, the Luddites among us won’t break the machines, but they will express skepticism about the quality of the content produced. Are machine-manufactured creations of a lower quality than those created by artisans? Of course, just like a mass-produced garment is of a lower quality than one that’s handcrafted. But who can afford handcrafted goods? The right question is whether consumers and producers can complete their tasks faster and cheaper than they can now. (Note 2: Examples)

And that’s the crux of this argument: The industrialization of content manufacturing is driven by the relentless pursuit of productivity advantages, not quality improvements. Once the industrial methods for manufacturing content reach a good-enough quality from the consumer’s perspective, the cost advantages will drive widespread adoption among consumers and producers alike.

This isn’t an apocalyptic vision, it’s just business. Productivity gains are steadily moving us towards consumer-directed, machine-powered content manufacturing. What many fail to realize is that Web 2.0 is far from the height of this model. In fact, it’s only the start. At what point will any consumer be able to push the buttons on the content-creation machines all by themselves? At what point will producers embrace industrial approaches and hand their customers the wheel?

So if not apocalyptic, this is certainly a disruptive transformation. Of course, highly skilled artisans are not going away, nor will their fine, handcrafted content. But demand will be affected, potentially marginalizing aspects of their services. However, this industrialization brings new opportunities, as well. Content professionals may choose to direct their skills to industrial methods, helping technologists design tools and developing innovative ways to include these tools in their services.

My colleague, Robert Barlow-Busch and I went to the Information Architecture Summit in Memphis last March to really dig into this topic. We presented an overview of how Web 3.0 technologies are disrupting the practice of information architecture (download slides, audio). There was also some excellent coverage of Web 3.0 topics by Chiara Fox from Adaptive Path and by Chris Thorne from the BBC.

So do people in the business of manufacturing content agree that we’re facing a fundamental change? Overall, I would say no. Many practitioners see Web 3.0 technologies as only tangentially relevant to what they do. They don’t view their work as manufacturing, at least not in an industrial sense, even as industrialists embrace these disciplines in the development of automated systems. (Note 3: Approaches)

But of course, if everyone saw this transformation coming, it wouldn’t be disruptive.

At the upcoming Web 3.0 Conference in New York, we’ll be keeping this discussion going. I’ll be participating in a panel alongside Eghosa Omoigui, Alex Iskold, and Greg Boutin, to discuss the state of Web 3.0 and Semantic Web technologies. If you’re in the area, I hope you can attend. And as always, please share your thoughts and ideas below.

Notes:

1. We’re measuring these productivity gains across a broad number of areas, including consumer participation rates in content creation, the amount of content created per contribution, customer value in terms of eCPM pricing for content created, and consumer value in terms of page views of content created.
2. A few examples of machine-synthetic approaches to content creation are listed below. Automated tasks include finding, selecting, describing, organizing, assembling, combining, and calculating content, all under the direction of consumers.
   • Google News: A computer-generated news site where the articles are selected and ranked by computers.
   • Kosmix: A “categorization engine” that organizes the Internet into magazine-style topic pages.
   • Wolfram Alpha: A demonstration of a “knowledge computation engine” that dynamically calculates facts in response to questions.
   • Primal Fusion: Example of a “thought networking” website created through a semantic synthesis technology.
3. There are many disparate fields that may intersect in the development of a Web 3.0 industrial system, including computer science, library and information science, cognitive science, interaction design, information architecture, and more.