A growing list of companies are making “interests” the focus of their value proposition: Twitter allows you to “follow your interests”; Gravity “unlocks the interest graph”; Pinterest “organize(s) and share(s) the things you love”; Quora “connects you to everything you want to know about”; Chime.in “connect(s) around your interests” — just to name a few.

Many believe the company that dominates interest networking will be The Next Big Thing (1) (2) (3).

But interest networking remains a bit of a mystery. What is it?

Interest networking is not social networking

Interest networking is often seen as an extension of social networking. Rene Reinsberg (MIT) describes the Interest Graph as “an online representation of individuals’ interests, with people and interests being the nodes of the graph.”

Part of the confusion here is that social networks are often leveraged to construct interest networks, but the end shouldn’t be confused with the means. By definition, a social network organizes information about people and their activities. An interest network organizes information around a set of interests, which may be yours and yours alone. Interest networks do not need social networking.

Collective vs. individual interests

Virtually all interest networking products and services are focused on collective interests, shared across groups of people. For example, Quora takes a collective approach, as does Chime.in, which lets users “build communities around topics.”

But the collective approach fails to recognize that interests are highly individual. It’s frustrating to have an interest in, say, classical guitar duet sheet music, only to be told, “Sorry, we don’t know anything about that. Did you mean Learn Guitar?”

Interest networking services need to manage the individual, unbounded nature of interests.

Transparent vs. opaque interest networks

Social networks don’t leave you with any doubt about who your friends are. Similarly, interest networks should make it transparent which interests they connect on your behalf.

Many companies are building black-box, opaque interest networks. When these services provide recommendations, they frequently have an almost-creepy feeling to them: We’ve recommended this article to you based on a statistical analysis of your usage. It makes the user wonder what the service knows about them.

Interest networking services should be absolutely transparent, enabling more human-like recommendations: Because you like the history and culture of Egypt, we thought you would find this article about Cleopatra interesting.

The Core Features of Interest Networking

In summary, interest networks should:

• organize around interests, for people, not about people;
• support very specific and individual interests, not just broad topics or categories;
• be transparent in their operation.

In 2012, Primal will begin rolling out our interest networking API with these core capabilities in mind. We invite you to join us in building interest networking services and applications, powered by Primal.

For more information, please read:

• http://about.primal.com/solutions/
• http://about.primal.com/developer/

At Primal, we believe software agents have a big role to play in the Web’s evolution. More like virtual assistants than applications, software agents work continuously on your behalf, delivering value even while you’re away from your computer or smartphone.

But a virtual assistant is only as smart as its data. Imagine you commissioned an assignment like, “keep an eye on the latest news, and email me when you find a story I will like.” How could you communicate your specific interests to your virtual assistant?

You might provide it with a phrase, say, “non-invasive cancer treatment strategies under development.” But without understanding the meaning of that interest, the assistant could only send you articles containing those exact words.

Primal is building a virtual assistant we call the Research Assistant that addresses these issues using our computational engine and semantic synthesis technology.

Here’s how it will work:

• You email the Research Assistant with a few words to describe a new assignment.
• The Assistant uses Primal’s computational engine to generate a rich interest network. In our example, it would include hundreds of topics closely related to your assignment, like “research”, “chemotherapy”, “leukemia”, “robotics”, “pharmaceuticals”, and so on.
• This new information greatly expands the amount of data available to the Assistant, giving it the ability to truly understand your assignment.
• The Assistant operates autonomously, analyzing everything from documents, news articles, and blog posts, to tweets and social media, looking for content that matches your interests.
• Whenever the Assistant discovers content it thinks you will like, it delivers it to your favorite RSS reader.
• The Assistant also pays attention to which articles you read and which ones you ignore, and uses that information to improve your interest network over time.

If you’re as excited about the Research Assistant as we are, then you will be happy to learn that we need alpha testers to help us build it. To participate, please visit research.primal.com.

Primal’s Mo Bros for 2011.

During November each year, Movember is responsible for the sprouting of moustaches on thousands of men’s faces around the world. With their “Mo’s”, these men raise vital funds and awareness for men’s health, specifically prostate cancer.

If you’d like to help out, please donate.

Excerpt from The Waterloo Region Record, Technology Spotlight 2011

By Chuck Howitt, The Record

Two years ago Tony Sarris was living the American dream. He was an engineering director for Unisys, a large U.S. information technology company with 37,000 employees worldwide. He lived in the west coast paradise of Laguna Beach, California. He made a comfortable salary.

Today, he works for a small Waterloo internet search company with 35 employees that few people have heard of. Goodbye surf, sand and sun. Hello grey skies, snow and cold winters. Has he lost his mind?

Sarris admits that at first he was hugely skeptical about moving to this area to work for Primal, which specializes in semantic technology to make web searching more personal. Now he has no doubt it was the right decision. “For the opportunity to work for this company, I would go to the Arctic, I think. Well, maybe I shouldn’t say that,” he says with a laugh.

For years, educators and leaders in Waterloo Region’s tech community have worried about a brain drain to the U.S. — losing the best and brightest to the lure of Silicon Valley or other tech clusters like those in Seattle and Boston. To be sure, that kind of exodus is still going on. Grads and co-op students are realizing the Canadian dream by moving to San Jose to work for technology companies. But a brain gain in reverse is also starting to awaken.

Companies in Waterloo Region are developing their own game-changing technology, and big league talent from south of the border is moving north to catch the wave. “A-players want to be challenged,” says Yvan Couture, chief executive officer and co-president of Primal. “We have an office full of top notch people who could work anywhere. They could make more money somewhere else. They work here because the project is hard and that’s fun.”

Sarris’s odyssey to Canada began two years ago when he was asked to be a guest speaker at a conference on semantic technology in San Francisco. Sharing the bill with him was Peter Sweeney, founder and co-president of Primal. Sarris had submitted an abstract on where semantic technology was going and what was needed to make it happen. He considered himself an expert in the field and had studied it since the concept first surfaced about 20 years ago. When he took a look at Sweeney’s abstract, his jaw nearly dropped to the floor. The little Canadian company was already working on much of what he was suggesting. “I was just amazed that some company was out there doing a lot of the most challenging pieces,” he says. “There were people working on the edge pieces, starting to nibble at it, but very few were working on the core of the next generation of semantic technology.”

The pair started communicating back and forth. One day Sweeney asked Sarris if he would consider working for Primal. “I said, ‘Canada? You want me to move from Southern California to Canada? It’s a beautiful place to visit, but I’m not living there’.” Nonetheless, Sarris came up for an interview and started looking around. He had heard of the University of Waterloo, but knew nothing about the community itself. He was surprised at how similar it was to some of the tech pockets in the U.S.

At the same time, he was bored at Unisys. He says that like a lot of large corporations, it was focusing on advancing the current state of the art, not breaking new ground. He missed the excitement of working for a small, innovative company. In June 2010, he moved north and hasn’t looked back since.

Sarris isn’t the only American with impressive credentials working at Primal. Nikhil Sriraman was employed as a patent attorney at the U.S. Patent and Trademark Office in Washington, DC, when a friend decided to leave the patent office to do some work for Primal in Washington. Sriraman was curious. What was it about this small Canadian company that would lure his colleague away from the patent office? He went to Primal’s website and started doing some research.

At the same time, he looked at its patent applications. “I thought, this is some really, really good stuff,” says Sriraman, who specialized in patents in robotics and artificial intelligence, an element of semantic technology.

He was so impressed that he offered to work for Primal for free in his spare time. He also came for a few visits. “The more and more I got exposed to the culture and people and technology, the more and more I kind of got drawn in,” says Sriraman. He moved to Waterloo last November to work as Primal’s full-time U.S. patent attorney.

In both cases, Couture says Primal set out to find the best people it could get, regardless of where they lived. “Tony probably has more depth in semantic technology than anyone in this company. On the patent side, the core patent market-place is in the U.S. so having a patent attorney from the U.S. made a lot of sense to us.”

Read full article…

Have you ever pondered how semantic engines, vertical search, and interest networking relate to enterprise mobility? In this podcast, Kevin Benedict interviews Primal’s founder and co-president Peter Sweeney on these subjects.

Summary: Big data technologies are plagued with small data problems. Their performance suffers in markets that aggregate a large number of unique interests. Some of the largest markets share these small data characteristics, including local ecommerce, personalized media, and interest networking. New approaches are needed that are far less sensitive to the cost and complexity of the data.

There’s a frothy enthusiasm for technologies that analyze big data for knowledge and insight. As search, social, and content networks are mined,  a global Web of knowledge will emerge.

While we share the enthusiasm, this vision needs to be tempered. Big data alone will not wrestle down the complexity of knowledge at Web scale. Anyone who has suffered through the unmet promises of local ecommerce, personalized media, or interest networking has felt the limits of big data first-hand.

The reality is that big data is plagued with small data problems. As our interests become more specific, the data available to describe these interests quickly thins out.

Social networks such as Facebook and Twitter illustrate the paradox of small data problems in big data. These networks are obviously massive: hundreds of millions of users creating profiles and media that encode innumerable interests. However, with respect to any specific individual or interest, the data is often sparse.

What follows is an explanation of the small data problem in big data and how it encroaches on many lucrative markets.

Why Big Data Fails

The Cost-Performance Barrier

In order to see small data problems in big data, you need to appreciate the shape of the data. Big data isn’t some undifferentiated mass. The shape of the underlying data has a tremendous impact on the productivity of knowledge modeling solutions. The figure below illustrates the shape of big data.

Domains of knowledge may be categorized along two axes: the breadth of data that encodes the knowledge (big data vs. small data), and the expressiveness or diversity of knowledge within the domain (complex vs. simple schema). Simple domains, such as lists of bicycle parts, are dominated by facts as objective, bounded knowledge; complex domains, such as politics, are characterized by their unbounded, subjective quality.

In a nutshell, big data fails when a knowledge modeling technology runs out of resources before the desired level of fidelity is attained.

In the figure above, this is illustrated as the cost-performance barrier. Three approaches for generating knowledge models are plotted along this cost-performance barrier: purely ontological (explicit) approaches, purely statistical (implicit) approaches, and hybrid statistical-ontological approaches.

Statistical methods thrive on big data. For these approaches, the resource shortage is in a lack of interest-specific data to mine. The quality of the model induced through these methods degrades as the underlying knowledge becomes increasingly diverse.

Manual approaches to knowledge engineering—such as building an ontology or database schema—face a different type of resource shortage. Within mass markets, methods that model the knowledge explicitly become prohibitively expensive. The knowledge becomes increasingly fine-grained and multi-faceted, surfacing ambiguities that are difficult to resolve. This increase in complexity imposes diminishing returns on the knowledge engineering effort and constrains ontological approaches to small markets.

Rarely are solutions either purely statistical or purely ontological. Any number of hybrid ontological-statistical approaches may be plotted along this continuum. Solutions often trade scalability and expressiveness, mixing statistical and ontological approaches to suit the characteristics of specific domains of knowledge. Critically, hybrid statistical-ontological approaches can make trade-offs along this cost-performance barrier, but they cannot break through it. There is a coupling between the expressiveness of the knowledge model and the efficiency of retrieval and reasoning systems. As the knowledge model becomes increasingly complex, the costs in leveraging those models increase in lockstep. There’s no free lunch.

Where Big Data Fails

Small Data Problems in Mass Market Opportunities

The cost-performance barrier leaves many lucrative markets inadequately served by big data approaches. The framework introduced above illuminates the common features shared across these markets that make them so challenging.

For ontological approaches, large collections of complex sub-domains within each area make knowledge engineering prohibitively expensive.

For statistical approaches, the relative scarcity of data pertaining to specific interests leaves large gaps in the resultant knowledge models.

The table below uses a few of the most lucrative markets to illustrate the challenges faced by existing approaches.

Conclusion

Big data is not the magic bullet many imagine it to be. These analytical approaches inevitably break down when confronted with the small data problems of our increasingly complex and fragmented domains of knowledge.

Cost is the barrier. Specifically, solutions must be devised where the knowledge modeling costs are far less sensitive to the complexity of the data. Although innovations within analytical approaches will continue to inch the cost-performance barrier forward, new perspectives are needed to tackle our most challenging domains of knowledge.

Most people want to track their interests without the hassle of deciding which people to follow. Primal Radar can help. It follows your interests and alerts you to new information and updates as soon as they become available.

We’ve added a number of features to Radar to provide a more personalized experience:

• Related interests: After providing a few words to start, Radar will surface topics and content that you can use as a starting point. Clicking on a topic in the navigation bar will narrow the scope of Primal Radar to that specific topic.
• Build an knowledge network: As you explore, new topics will be automatically added to your knowledge network. Primal uses your knowledge network to understand what interests you, providing more personalized results over time.
• Explicit personalization: Review and modify the content of your knowledge network at any time by clicking on “Top Thoughts” or “All Thoughts” in the navigation bar.
• Add and remove topics: You can add new topics to a page by typing them into the navigation bar. You can also delete the ones you no longer want.
• Social sharing: You can share a feed of your interests with others, either via RSS, email, or your favorite social network. 

Primal Radar is free for individuals. Give it a try and let us know what you think. If you need more information, please visit the Primal Radar Learning Center.

We were recently featured in The Waterloo Record for “Planking”. Check out the article today!

It’s Friday afternoon, and your boss asks you to write a report on a relatively obscure topic. Soon after starting you realize there is no way the report will be ready on time. You try some Web searches and are quickly overwhelmed with irrelevant information. What you really need is for the report to meet you halfway.

Primal Pages can help. It summarizes information from the real-time Web and organizes it into a personal website. Pages assembles your website on-the-fly as you explore your interests. You can use it to quickly outline the content you need instead of searching across the Web for it!

We’ve added a number of features to Pages to provide a more personalized experience:

• Create content: Clicking on topics creates content instantly and in an organized format.
• Real-Time Web: Explore topics, related topics and content from the real-time Web that interests you by clicking on terms in the navigation. Clicking on a term will change the content of your Pages and make it increasingly more relevant for you.
• Build a knowledge base: Every time you click on a term, Primal will add it to your knowledge-base of topics. Pages remembers your interests to better anticipate your needs over time.
• Personalize: You can add new Pages by adding in new terms in the navigation menu. Primal Pages lets you “pin” information on your page, delete or add terms, and add your own introductory content.
• Topic Suggestions: Pages will surface suggested topics and relevant content that will enhance your knowledge-base and allow you to deepen your understanding of a topic.
• View original source: Content on your site has the original sources available for you to visit at any point.
• Dynamic links: Highlight new topics within the content and generate links to new Pages.
• Publish: If you like, you can generate a unique URL for your Pages that you can share with others. Visitors to your Pages will see the same selection of topics and whatever content you’ve created or pinned.
• Social: Share your Primal Pages with your social network via Facebook, Twitter, or email

Primal Pages is available free for individuals. Give it a try and let us know what you think. If you need more information, please visit our Learning Center.

In the early days of the Web, librarians would often compile directories of “trusted sites” on a range of important topics. In fact, Yahoo! has its roots in what was essentially a human-edited directory of online content — then called David and Jerry’s Guide to the World Wide Web. These are early examples of online content curation.

A content curator is someone who continually finds, groups, organizes and shares the best and most relevant content on a specific issue. (link) In the past, curating online content was a time-consuming, highly manual process; few people did it. It was largely the domain of small groups of professionals.

Today, new services like Tumblr and Posterous have begun to attract part-time, amateur content curators by making it easier than ever to “publish” the interesting things you find online. While these services have undoubtedly brought content curation to new audiences, the time and effort involved remain significant barriers. The vast majority of people continue to stay on the sidelines.

Before content curation becomes a truly mass market activity, further technological innovation is needed to lighten the burden of the entire process of finding, organizing, and grouping, and sharing content.

Taking the work out of content curation
One problem with existing tools is that they force people to work at the level of individual pieces of content. Given the flood of content on the Web, this approach requires a lot of manual labor on the part of a content curator. If the manual approach is the only one available, content curation is doomed to be a niche activity.

A new system is needed — one that puts raw computing horsepower in the hands of content curators to help them get the job done orders of magnitude faster than they can today.

Content curators want to spend their time focusing on their editorial vision, pulling together the ideas that matter most to them and their audience. They likely don’t want to wade through hundreds of pages of search results and feeds, painstakingly organizing what they find.

The key, then, is to enable content curators to quickly express their ideas, indicate a few sources to get content for those ideas, and let computers do the hard work of finding and organizing content around those ideas. The final step is for the content curator to vet the work of the computer, modifying the results as they see fit to give the results that “human” touch.

With a product like this, content curation could become easy enough — and fast enough — that anyone could do it.

Using Primal to do content curation
At Primal, we’re building one such product. Try content curation using Primal Pages:

1. Visit think.primal.com and enter the topic you’re interested in.
2. Type in subtopics that interest you, or use the suggested subtopics from Primal.
3. Click “Read your thoughts” to have Primal do the heavy lifting of finding and organizing good content from around the Web.
4. Review the results, “pinning” content nodes that are especially appealing to you.
5. Click “Publish” to begin sharing the page on your favorite sites like Twitter and Facebook.

Happy curating.