1. Create a vocab or profile syntax that lets you give short (non namespaced) names to ontology classes and properties.
name = foaf:name
label = rdfs:label
where = is like owl:equivalent*, don't care about the syntax of this file at all, even RDF using owl:equivalent is just fine.

2. Create a way to reference this from an rdf file, something similar to profiles.

@proxy http://ex.org/my-proxy-ontology .
@prefix : .

:me a Person; name "Nathan"; nick "webr3"; birthday "31-03" .

and just to detract from this, I'd actually really quite like this syntax:

@proxy http://ex.org/my-proxy-ontology .

:me a Person; name Nathan; nick webr3; birthday 31-03;
knows (name Michael Hausenblas), (name Melvin Carvalho) .

literals shouldn't need enclosed in quotes unless they contain syntax grammer (although \ delimiting could cover this), it'd also be nice to be able to talk about the uri you just mentioned in context, and also to leave out obvious stuff like "a foaf:Person" on who you know, cos of course they are a person. Further, literal types could easily just be inferred by the predicate, and further still the ontology of the predicate should/could contain all the rules to validate the value using owl data type restrictions. (note, doesn't this cross over with xsd restrictions?).

just a random snippet of thoughts / brain dump!

What do you like about RDF?
The triple, (dereferencable) URIs as identifiers, use of ontologies/schemas, ^^typed literals, various different serializations.

What do you dislike about RDF?
documentation is pretty much serialization specific (RDF/XML) should be serialization independent
reification, bags and sequences in the rdf/xml specification
subject and object definitions do not match (lack of literal subjects, lack of collections in the subject position, lack of graph literals)

Most Important Addition to RDF
subject and object both taking the same values (literal subjects), graph literals, collections in the subject position, and variables allowed even if not supported by specific or existing serializations.

Tidy up RDFS and add in much needed values which currently reside in owl and a few other ontologies - RDF should come with a base vocab that covers most rdf specific needs.

Priorities
* Make RDF smaller/simpler: [ No opinion ]
* Make RDF larger/more powerful: [ No opinion ]
* Redesign some parts of the RDF/XML syntax : [ No opinion ]
* Redesign some parts of the RDF model: [ 5 +++++ (highest) ]
* Improve RDF's suitability for data/database work: [ 1 + (lowest) ]
* Improve RDF's suitability for semantic/KR work: [ 1 + (lowest) ]
* Provide better syntaxes for RDF: [ 5 +++++ (highest) ]
* Provide better documentation: [ 5 +++++ (highest) ]
* Explain the business case better: [ No opinion ]
* Provide better communication, community support: [ No opinion ]
* Help people find or develop RDF vocabularies: [ 4 ++++ ]
* Develop more compelling applications: [ No opinion ]
* Develop standard vocabularies: [ 5 +++++ (highest) ]
* Work on RDF Security: [ No opinion ]
* Work on RDF Trust & Provenance: [ 3 +++ ]
* Work on Synchronization, Distribution, and Versioning of RDF Data: [ 3+++ ]
* Work on bridging between RDF and XML: [ 1 + (lowest) ]
* Standardize RDF API in Javascript: [ 5 +++++ (highest) ]
* Standardize RDF APIs in other languages: [ No opinion ]

re: Improve RDF's suitability for X, if you improve the model this should happen automatically.

re: Make RDF smaller/simpler/larger/more powerful, again this is conflating issues, RDF should be looser so that serializations can tighten up by supporting certain features.

re: trust, provenance, synchronisation, distribution, versioning - imho this is all out of scope, work could begin on this if the model was loosened up to be more n3-like (graph literals); get the model right and these headaches become much easier to handle.

re: backwards compatibility, this should be a non issue as any looser definition of RDF can be countered by defining existing serializations as including a subset of RDF's features - to limit the next decade based on the mistakes or lessons learned from the last decade is, imho, unethical.

Add Core Support for Working With Multiple Graphs
graph literals, anything else is a work-around imho. if graph literals +5 in every respect, else, meh.

Create a Standard JSON RDF Syntax
critical imho, unsure if it should come under this working group though.. but if it's the only place and time then here it must be, as RDF/JSON is the most important serialization for the next decade of the web, easily.

Make Turtle a W3C Standard
great yes, but only after getting the core model sorted out, tweaking turtle to handle the changes (should be v easy given n3 heritage) then standardize under separate cover if possible - else, if not possible under separate cover, then imho must happen - the "why not" train of thinking comes to mind. Turtle *should* already be a standard.

Indicate Which RDF Features Are No Longer Best Practice
either deprecate them or leave them be, "weak deprecation" is nothing more than politics; keeping them for BC and allowing a new generation to use them a poor choice imho, deprecate them, mark them as deprecated, then if implementation want to be backwards compatible they still can (and probably are).

Extend RDF/XML
would much rather see RDF defined without any changes to serializations, let serializations conform and revise under separate cover.

Revise Semantics for Blank Nodes
this issue is imo nothing to do with RDF (indicates a problem in sparql).

Create Standards for Deployment of Linked Data
great idea but nothing to do with RDF core imho and would add way to much scope. can't rate it as a +5 because I view it as orthogonal.

Define Some Useful Similarity/Equivalence Properties
100% behind this one, as per J Hendler's RDFS3 proposal - nigh on critical to the wider community moving forwards.

Define a Namespace Packaging Mechanism
personally view it as a bell and whistle proposal, v low priority imho - however loosening the rdf spec to all namespaces to be either included or pointed to in order to allow future work like this would be a good idea.

Change RDF Semantics to Plain Data (SPARQL) Style
I'd just remove this from consideration if possible.

Explain How to Determine What a URI Means
? give something a uri, describe it with RDF, consult that RDF to read it's description, meaning is different in every context and.. unsure why this is on the list tbh.

Allow Literals as Subjects
+5, must happen imho.

Improve Integration with Syndication Systems (Atom)
it should be easy enough, indeed it's already possible - but syndication should and could be done with RDF, lack of graph literals pretty much makes RDF a no-go area in the message space, which is sad..

Other Comments?
IMHO, TimBL (http://www.w3.org/DesignIssues/RDF-Future.html) and Jim Hendler (http://www.w3.org/2009/12/rdf-ws/papers/ws31) have everything covered in their proposals, I completely fail to understand why these two proposals aren't just done as they cover everything and are imho golden.

If I could click my fingers and decide what happened, I'd get everybody working on making the changes outlined by TimBL and Jim Hendler, then get a group on to doing RDF/JSON under supervision of Sandro Hawke, Manu Sporny and possibly Jeni T. Get Turtle aligned with the changes (should be an easy hit) then clean up RDF/XML and define it as supporting a subset of RDF. Quite sure this won't happen though and I'll be completely confused as to why not.

As we all know, one of the biggest hindrances to innovation in the various areas of computer science is the human brain, and specifically the tendency to think inside, rather than outside the box. There are things we never even consider or imagine because we just can't think outside of what we already know/presume - a small percentage of gifted individuals can, but on the whole, we as humans/geeks/programmers cannot.

To keep this short and simple, let's return to the aforementioned point, with javascript, any libraries you write can be used on both client and server. Great eh, write an AMF parser and it works on both client and server -wonderful.

Now, let's flip the example to a 'shopping basket', you see what we did there? now you've got an ecommerce application on the client side (think about that for a minute, you could put items from any 'website' in your basket, not one, any/all - oh and keep http stateless, and, and..) - here's another one, an authorization protocol like openid or webid - so a server could 'login' to a browser.. - how about an HTTP server in js, yes now it's on the client side and the server side - and when each side is both client and server isn't that just p2p?.

This universality breaks down the barriers of convention, it gets rid of that wall we have between server and client - can you imagine how many 'server side' technologies, libs, patterns and architectures nobody has ever even considered working with on the client side, can you imagine the implications and the period of innovation this will bring on?

Conversely, can you imagine how many client side technologies and libs nobody has thought about putting on a server? fancy using HTML as a data storage serialization and editing it via jquery on the server perhaps?

Maybe you can't imagine, maybe I can't - but fact is it will be possible, and people will start doing it, often just by lack of these assumptions and notions the current generation(s) of programmers have, they won't assume that something we've always used on the server side will be on the client, they'll just do it because it makes sense, and we, well we'll all be astounded.

Make sense?

There have been some nice examples of combining client and server side js recently, and node knockout is sure to provide many more real soon - if you don't know of node.js I'd recommend taking a look. Also worth noting the realtime client push service, Pusher (for HTML5 WebSockets) - of course there are also node implementations, and here's a nice article with some more details and examples.

I've also been playing with some of google's projects (radar check, you know about the ajax playground ya?), namely Fusion Tables (here's one) which has a full REST API, infact that's what it is, a kinda web db merged with csv merged with RESTful goodness - and you can use them with V3 Maps which gives some really sweet results - like this heatmap of designated beaches on the coast of Brazil. Convergence check, you noticed that protovis can be used with google maps ya?

If there's one app I wish google would opensource, it's Google Moderator for an example see the one for TipJar. Btw, did you see 3D Lego Star Wars running in Chrome with HTML5 (credit to Unity 3D truth be told) - oh, and if you ever wondered how those 3D Lego videos we see on youtube all the time are made, you'll be looking for this.

Generally I'm really admiring ushahidi (projects on github), and especially Swift River and SiLCC - there was a good post on 10 ways to use Swift River which is certainly worth a read. Related, but don't ask why, here's a good post on prediction services which has a few good links. Crossover check, remember earlier I mentioned Fusion Tables w/ Maps V3 and Protovis.

Digital Bazaar have been working on a Javascript implementation of TLS (part2) namely Forge - code on github -explaining why this is such a big deal is outside the scope of this post, but I'd encourage you to look at PaySwarm and Digital Bazaar's fantastic blog which'll give you a good overview of what they're up to. Related and similar, it's worth noting jsSHA - a JavaScript implementation of the entire family of SHA hashes, random.org for all the true random data you could need, Bitcoin a peer-to-peer network based digital currency and amf.js a js implementation of AMF. Also there's the HTTPS Everywhere extension for Firefox.

And in non technical land, I've really been enjoying How to be a Retronaut, it features amazing media from the past, such as Colours of the Rothschilds… which amongst other things has a colour photograph of King Edward VII, taken in 1909 - other recent favourites include the Eiffel Tower being built, the first ever music video (1895) and the instant eye of Jacques Henri Lartigue.

If you haven't seen Hans Rosling's talk on global population growth go watch it now, v recommended - and finally, they figured out what came first, the chicken or the egg..

Here's the setup, a blog post which is a review of a new music album, the post has the familiar facebook 'like' button on there.

Here's the problem, my partner finds that people will only click the 'like' button if they 'like' the album, it's not about the post, the site, the quality of the writing in the review. All points to something wrong in the kudos chain.

It's not just like though, let's introduce a simple '5 Star' rating system on the posts, just what are the users rating, the post, or the album?

Now, let's change the example somewhat, scenario: a well write article about a horrific genocide - hundreds of people saying 'I Like this', as humans we can quickly infer (and somewhat hope) that people are saying they like the article, and not the genocide, but what about a machine.

For your consideration, each web page typically includes multiple distinct things, so we need a way to be able provide users with a way to do 'stuff' which each thing, I like this review, I like this author, this topic is worth noting, I like the primary topic, I've read this and so forth - any semantic web readers will quickly say "ahh problem solved! give everything a URI" (I hope) - but the problem isn't solved, it brings up (yet again) the issue that we need a whole new generation of UX/UI improvements, one 'like' button will not do, when you've got 10-100 things on a page, and different actions for each, this is something that's clearly going to have to be abstracted out from the page and handled in a different way - just how I don't know..

Will leave it there, you can see the challenge..

The specific issues are:

• Read-Write Linked Data
• Socially Aware Cloud Storage
• Levels of Abstraction: Net, Web, Graph

I'm yet to disseminate all that's changed, but they certainly are filled out and refined, remember folks the devils in the details!

Quite sure that I'll follow up with a bunch of notes, as will a few others - but for now, there's the heads up that it's time to do a bit of reading.

In other words, you can now use the fantastic tabulator code on the client side, and on the server side - the released code is just a quick port to allow you to get hacking, I'm not planning to maintain it, because I feel it would be far more beneficial to make the rdflib.js code work on both client and server from a single library rather than forking it and loosing the hard work of all involved.

rdflib.js ported to node (was incredibly easy, only had to change a few lines)

A quick example (parsing n3 foaf file and showing everybody with a foaf:name):

var sys = require('sys'), rdf = require('rdflib');
var rdfdata = ''; //some rdf in n3 here; probably a foaf file would be a good idea
var kb = rdf.N3Parser( new rdf.IndexedFormula() , null , uri , uri ).loadBuf( rdfdata );
var FOAF = rdf.Namespace('http://xmlns.com/foaf/0.1/');
var results = kb.each( undefined, FOAF('name') );
for (var i = 0; i < results.length; i++) {
sys.puts( results[i] + ' ' + kb.the( results[i] , FOAF('name') ) + '\n' );
}


Related, I've also updated Sean B. Palmer's FOAF example to work on the clientside with RDF/XML and with N3.

Back to node.js stuff, I've also released a patch which exposes all the details needed for FOAF+SSL in node to node_cryto.js (hopefully it'll get added to the main git repo v soon) - next step is to get a JS FOAF+SSL library working, which shouldn't take too long.

v8 and server side ecma-262 (javascript) is a fantastic thought, and definitely the future imho (universal language).

To get going and start making server side js web applications there are two good approaches, v8cgi which aims more at letting you run js programs as a cgi app (like you do with php cgi) under apache and similar, seems to be picking up speed, nice approach (workable) - the other is node.js which is off to a great start as a pure server side environment where 'HTTP is a first class protocol' which just proves what a good start Ryan Dahl has made.

Briefly to touch on v8, it allows you expose C code to javascript, which is.. well perfect, would be good to see far more lib's exposed to js using tools like cproxyv8 and v8-juice - in particular I'm looking at you librdf (and an interesting sub-thought, apache and openssl).

Back to node.js, seems to be exciting already has a strong API to get some great demo's and projects off the ground, and very well aligned with the W3C work (or maybe what-wg..), specifically on the Web Worker API front; in fact there are already implementations of Web Sockets for node.js (and socket.io) - nodelint (jslint for node) is a handy addition, whilst some projects seem to be going all in, like Persevere.

As for adapting current code to work on the server side, you'll want to be looking in to the modules .. which.. well to be honest that's a new link I just found, and not what I was thinking off.. emm, I think that provides almost everything I need - might be about to jump back in with two feet!

The rest of this post was about to mention php.js, harmony framework (Converts PHP 5 code into Javascript), and then mascara and of course haxe which compiles to js amongst many other targets.

Might have to come back to this post.. for anybody who follow what I do, thus far I'd figured out that node.js is http+tls friendly, you can get a clients certificate (for foaf+ssl) and you can send application webid+ssl certs too, although up till now I hadn't found a way to generate certificate (let alone using spcak) but that could be wrong..

This is quite a dry subject, and I could be wrong (in fact in some ways I want to be proved wrong, this is how we learn), but do read on if you're interested.

Example

Over the past few months I've hit on a number of occasions where I was convinced I needed Named Graphs in order to address the task at hand.

A notable example is the scenario where using WebAccessControl and the ACL ontology, a system would have to figure out just who should be given access to a resource, and who should be denied.

In this example I'll cover the notion of ACL for "groups" in a linked data world.

The task at hand is to allow access if:
the graph serialized within the document obtained by dereferencing the URI of the group states the <webid#me> is a member.

Otherwise written as:
if we dereference <groups#admin> does the graph returned include the following { <groups#admin> sioc:has_member <webid#me> }

Or in SPARQL:

ASK
GRAPH <groups> {
  <groups#admin> sioc:has_member <webid#me>
}

In this example we *do not* want to dereference the users webid to see if the graph returned specifies that { <webid#me> sioc:member_of <groups#admin> } , or indeed consider the open world possibilities that another yet unknown graph could assert that the user is a member of our admin group, as that would breach security.

The ACL

To proceed with the example, consider the following ACL:

[] a acl:Authorization ;
	acl:accessTo <https://example.org/sensitive> ;
 	acl:agentClass :mygroup ;
 	acl:mode acl:Read .

:mygroup owl:equivalentClass [
 	a owl:Restriction ;
 	owl:hasValue <groups#admin> ;
 	owl:onProperty [ owl:inverseOf sioc:has_member ];
 	] .

The Problem

The problem proposed by this ACL is that any of the following four sets of triples would infer that <webid#me> would qualify as an instance of :mygroup (or a member of <groups#admin> if you prefer).

• <webid#me> sioc:member_of <groups#admin> .

• <webid#me> _:x <groups#admin> .
  _:x owl:inverseOf sioc:has_member .

• <groups#admin> sioc:has_member <webid#me> .

• <groups#admin> _:y <webid#me> .
  _:y owl:inverseOf sioc:member_of .

In other words, the ACL does not specify a "Named Graph" to query, and at the moment, no way exists to specify with (OWL or RDF) which "Named Graph" to query / trust.

This, point in case, is one example where I saw the need for Named Graphs in RDF and OWL.

Another way of looking at it

You will have noticed the notion of "Named Graphs" creeping in above, seems like a logical thing to say, especially when you consider that to process this ACL and grant access you'd probably use SPARQL, and specify a Named Graph to query over. However, much of what follows arose because I'd decided not to use SPARQL, and rather to code an ACL processor in my preferred language.

If you consider the situation, the ACL processor which decides if access should be granted or not, must implicitly "trust" the document which contains the serialized ACL graph. That is to say, that it must by extension trust any resources pointed to by said ACL, and if it doesn't then the ACL isn't fit for the purpose.

It's also important to note that "trust" is context specific, in this case we trust the resources pointed to by the ACL for the purpose of WebAccessControl.

One could then pretty quickly conclude that in this scenario the ACL processor already know's how to process the ACL, it must only use resources it trusts, therefore it must only allow access if the graph serialized within the document obtained by dereferencing the URI of the group states the <webid#me> is a member.

(because <groups#admin> is specified in the ACL, and thus by extension, trusted)

Named Graphs in SPARQL

The aforementioned logic would also apply if I was using SPARQL to process the ACL, it would equate to the ACL processor asking:

ASK
GRAPH <groups> {
  <groups#admin> sioc:has_member <webid#me>
}

But again this is very context specific to the example, let's consider for a moment that the URI for the group could have been a non-fragment URI, <groups/admin> for example.

This leads us to an important problem, when we dereference <groups/admin> it would have to 303 See Other through to a different URI, let's say <data/groups/admin> - which would then mean that the Named Graph to be used was <data/groups/admin> - this URI, you may note, we do not know when we are writing our ACL; so if we ASKed the above SPARQL, the results would always come back negative, since their is no GRAPH <groups>.

The URI of the Named Graph issue is compounded by modern web servers and publishing practises, because <data/groups/admin> could easily be content negotiated (or rewritten), thus giving various final URI's of <data/groups/admin> or <data/groups/admin.rdf> or <data/groups/admin.ttl> or <data/groups/admin.n3> and so forth. One could quite easily (and often does) end up with the same Graph repeated multiple times within a quad store, all under "different" "Named Graphs".

I'll expand on a possible way of addressing this problem further on.

Directionality

Previously I mentioned that the ACL processor didn't have a problem with the above ACL, because it by nature trusted all resources which were mentioned in the ACL graph. However, again this is very context specific.

Let's consider for a moment an inverted ACL, where we want to allow access if:
the graph serialized within the document obtained by dereferencing the URI of the users webid states that <webid#me> is a sioc:member_of <groups#admin>.

We don't know the users webid ahead of time when we write the ACL, so again we have no way of writing how to trust a resource - it is critical to note that even if RDF(2) did support the concept of Named Graphs, it still wouldn't address the situation because we wouldn't know the Named Graph ahead of time, in order to trust it!

If we now consider the following ACL:

[] a acl:Authorization ;
	acl:accessTo <https://example.org/sensitive> ;
 	acl:agentClass :mygroup ;
 	acl:mode acl:Read .

:mygroup owl:equivalentClass [
 	a owl:Restriction ;
 	owl:hasValue <groups#admin> ;
 	owl:onProperty sioc:member_of;
 	] .

The outcome of our previous logic concludes that again we should be querying the "trusted" resource <groups#admin>, which gives us another problem, that's not the resource we want to be asking in this scenario.

The only thing that remains, and I'll later argue the only thing that ever matters in a web of linked data, is direction.

If we analyse the first ACL closer, we can see that we ultimately used the direction inferred by the presence of owl:inverseOf to place <groups#admin> in the subject position, rather than the value/object position it could have been in, indicated by the presence of owl:hasValue. (bare with me).

In this example, we can use the strong semantics of owl:hasValue (and lack of owl:inverseOf) to place <groups#admin> in the value/object position, and thus our ACL processor can come to the outcome we want, which is to look for the a triple with the meaning { <webid#access> sioc:member_of <groups#admin> }, and that means dereferencing the URI in the subject position, in other words asking the graph serialized in the document returned by GETting <webid> if it contains such a triple.

I've applied some understanding to OWL that quite simply isn't there though, as I earlier stated both ACL examples could easily equate to looking for any one of those four sets of triples.

However, this is the point - machine understanding of data is in the domain of the machine, the application doing the processing. And "truth" or "trust" is entirely context specific.

I'm increasingly convinced that the combined context of the data in a graph and the context under which that graph is being queried, specifies or infers in which direction you want to be reading, and directionality can be determined with linked data by dereferencing whichever uri you place on the left / in the subject position.

I recently found that Tim Berners-Lee wrote about this in a blog post entitled Backward and Forward links in RDF just as important:

One meme of RDF ethos is that the direction one choses for a given property is arbitrary: it doesn't matter whether one defines "parent" or "child"; "employee" or "employer". This philosophy (from the Enquire design of 1980) is that one should not favor one way over another. One day, you may be interested in following the link one way, another day, or somene else, the other way.

Key here is the sentence "One day, you may be interested in following the link one way, another day, or somene else, the other way.", and that is exactly what all these examples are doing, following a link one way, or the other way.

To conclude this part, in every scenario thus far where I've thought I needed Named Graphs, it turns out that I in-fact needed directionality - and because I'm dealing with Linked Data, whatever I place in the subject position defines the URI which I need to dereference, and ultimately the Graph(s) which are considered when resolving the answer to the question being ASKed.

I'd thus suggest that "Named Graphs", do not exist in a web of data, they are needed in N3 and when using rules, because all data is often in a single file, however that is not the case for Linked Data, where we dereference.

Back to SPARQL and Named Graphs

Previously I mentioned the complications with the way we currently use named graphs in SPARQL and in our quad stores, where the URI we end up using could literally be, anything; and often we get duplicate data under different graphs.

To address this, I'd suggest that what we should be storing as the graph ?g value, is not some made up "named graph" but rather: the dereferenced URI which we initially requested.

• in the case of <group#admins> this would be <group>.
• in the case of <group/admins> this would be <group/admins>

To clarify, *never* the URI that a GET request finally resolves to, and *always* the initial dereferenced URI we requested.

The above ensure that we'd never have duplicate data in our quad stores again, that SPARQL queries including a FROM clause always dereferenced, that publishers and web server administrators were free to relocate and restructure their data, and ultimately make for a much nicer, healthier web of data.

Cool URIs don't change, and they wouldn't, just because the final document serializing a graph may move to a different URI, doesn't mean the original URI has to change.

Conclusion

Apologies for the length of the post, but I figured everything needed covered, in context. Simply put we need to focus less on Named Graphs (which IMHO aren't needed) and focus more on directionality. Every problem I've encountered thus far is covered by what Tim said years ago: "One day, you may be interested in following the link one way, another day, or somene else, the other way."

Comments?

Something to try

Next time you start an application, why not create a subdomain for your data tier, create a script for each SQL query and call it via HTTP (honestly you won't notice a speed difference), similarly expose all static files your system uses on the same (or a different) sub domain. Here's the advantages:

1. You can scale without changing your application
2. You can swap servers without changing your application
3. You can move data around your server and take advantage of all the HTTP servers features.
4. You can cache your resources and utilize HTTP caching.
5. You can distribute or relocation your application anywhere on the net without worrying about data.
6. You can migrate over to different data systems (swap database vendors etc) without changing your application.
7. You can call every resource on the web in the same manner, from APIs through to linked data and all in between.
8. You're critical SQL and data mapping code will be fully abstracted and in self contained files (ultra easy to edit, bug fix, maintain).
9. You can take advantage of HTTP logging, and HTTP status codes instead of custom exceptions.
10. You have a ready made RESTful API in to your data tier, so you can hook on other applications, or open it up to third parties, or the entire net.

So much more, all by one simple step - and that's only the first half of the cartoon ;)

Generations One and Two

Historically we have been through two generations of the web so far, not web 1 and web 2.0, but rather the first round of mounting the presentation tier on the web (static publishing documents and media), then the second round of mounting the application tier on the web (from forms through to the current api's we see everywhere).

The next round, and what many are currently hacking at, is mounting the data tier on the web, mainly via the "Linked Data" movement which you may have noted.

This third round is critical for two primary reasons:

1. By mounting the data tier on the web, it allows us to remount the application tier again, but this time properly, where HTTP is the API between the Application tier and the Data tier. (true REST)
2. The "Internet of Things" is very much dependant on the "Things" speaking the same language and sharing a single Data Model, where the model remains the same but the vocabulary can change.

Generation Three

"Linked Data" is in many ways the solution to the above, because it has a Universal Data Model (EAV), a Universal API (HTTP), and Universal Addressing & Identification (the duality of http URIs) - more importantly though, "Linked Data" provides the means to mount and expose both ABox and TBox statements on the web - this is the key detail where RDF wins over other EAV based data models; the ability to store both the data and the vocabularies on the web, and both in the same manner.

Linked Data is currently tied (but not bound) to RDF. However, RDF still has very critical limitations, primarily it doesn't have any notion of Named Graphs or Quads. Trust and provenance are essential moving forwards (with quads you can select which data your application trusts, and which it doesn't, based on the source, the Named Graph - without it you can't), as is the ability to transfer data from multiple sources at the same time (update streams, batch operations, merged data with provenance - again needing quads).

The above means we need an RDF2 (or standardisation & adoption of N3+rules), and moreover we need standardized non-xml serializations of said RDF(+2), the most important being rdf/json.

This brings me to HTML5, the key here isn't the new HTML5 document format, the semantics, the ability to embed microdata or anything like that - it's the introduced (or implied) dependency on JS /ECMAScript via the JS APIs. This leads to JS being rolled out on to the majority of devices - it's also important to note the server side JS movement in this too.

Generation Four

We've had a "Web of Documents", we're building a "Web of Data", the next logical step is to have a "Web of Applications", for this we need two things: a Universal Programming Language (JS...) and an "Internet of Things" which support the universal data model and the universal programming language.

As far as I'm concerned HTML5 plays a critical part in the big picture:

- Short term, it at least unites the browser vendors to better support the techs which developers use.
- Presently (and from here on) it allows us to start hacking at the web of applications.
- In the future it's legacy will be it's introduction of JS as a Universal Programming Language.

Sides:
It stands to reasons that RDF/JSON (or whatever supersedes RDF serialized in JSON) will also play a major part in this round.

We may well see a shift from client-server, server-server to application-application; where each machine on both sides comprises of HTTP Client, Server and Cache.

Generation Five

Leading on from here we get to the fifth round of the web, remounting the Presentation Tier, but this time where the presentation tier speaks to the application tier through HTTP, the Universal Interface.

HTML (together with JS and CSS) again plays a critical role in round five, because up until this point it remains the only way for the hackers to create this fifth generation, notably it will be a lot more than we are have now, after years of maturity, hacking and support universally. Naturally then we can assert that HTML will remain the core of the Web's presentation tier so long as the Web exists.

Additional Shifts

Widespread adoption and understanding of REST, truly without this we'll never get past generation three.

Data transformation, for a very long time data transformation will be an everyday service we all require, for legacy systems, legacy data, proprietry systems, data model conversion and so on - the key isn't to be able to understand all kinds of data / serializations / models, but to transform it in to what you do understand, easily.

The enabling of the Semantic Web, Linked Data provides the means to do this, it is the enabler - each EAV triple we put out is also a triple that can be reasoned over, full machine understanding of all our data is an ultimate goal that will take many years. Generation three very much pushes the semantic web in to the realms of the real world, no longer for the strictly academic.

The great cleanse, all the data from generations one through three (and possibly beyond) will need cleansed - as primary universal vocabularies emerge the task will be to clean what's on the web and migrate it in to the new models, reasoning and cleaning it as we go.

Cross compilation and seperation of syntax from machine code - another movement that is gaining momentum, no longer are we tied to a specific syntax for a specific JVM - not far off is the day where we can program in our preffered language, and compile it to whatever target we need. This is all ready happening in many quarters (like haxe) and soon will be the norm.

Seperation of human readable data syntax and data storage/transfer serialization. This has long been understood, however with EXI comes a light serialization of XML for the machine & transfer; logically from there we can expect the same in the future for RDF(2) and most data moving forwards.

Much of the code needed for the Application Tier will disappear over time, typically most application code involves taking some data and turning it in to some new data, processing it - when you can do this on the fly via data transformations, rules, reasoning, inference, querying and similar, you remove the need for most f the code in between. It is entirely possible that most server side web applications could disappear.

End

It is testament to the strength and design of the web that it can support every incarnation of the web thus far and in the future all at the same time - All credit to Tim Berners-Lee, Roy T Fielding and the many, many others who contributed, evolved and continue to forge this thing we call the web.

Disclaimer
Obviously, this is all just my humble opinion, none of it is fact - but from stepping back a bit this is what I conclude, for now.

I've skipped loads of things, most of the techs I work on and think about daily, FOAF+SSL, web access control, read write web of structured linked data, loads more - the above is just a summary of where I think we're headed, primarily for my own reference :)

Universal doesn't necessarily mean universal - but it's a good enough word to convey what I'm thinking. Any other use of terminology you don't quite agree with, likewise, terminology is not the point of the post ;)

Best & happy to hear your thoughts and additions,

Nathan

OpenCalais and Zemanta usage details and service comparison.

The demo leverages OpenCalais in order to pick up references to things, which are returned in most cases as string literals; OpenCalais can also be configured to return back socialtags which give a broad stroke idea of what the document is about, again with string literal "tags". With regards the references (semantic metadata, Entities, Facts, Events etc.) which OpenCalais returns, whilst it is generally string literals, it also returns back vital Type and Relevance information, so in the case of "London" it will also assert that London is a City. Even in the case where it doesn't previously know what a thing is, it can work out that say "Frank Neverbeenheardofbefore" is a Person.

Zemanta is also leveraged, the primary difference between Zemanta and OpenCalais (and thus the need for both services) is that Zemanta focuses more on accurate tagging of text. Primarily though, Zemanta tags (again string literals) are meaningful tags which are commonly known and are referenced to either existing Linked Data identifiers such as http://dbpedia.org/resource/London and further information about the tag (or thing), in the case of the aforementioned London, then it will often also provide links to the wikipedia page for London, the official homepage to the city of London and a link to show the position of London on google maps.

I should point out that ever increasingly OpenCalais also returns back Linked Data too, for instance in the case of London they have given it an HTTP URI which can be dereferenced to retrieve more information about London. At a very crude estimation I would suggest that (depending on the subject matter) OpenCalais returns Linked Data URIs for about 15% of all references it finds to well known "things".

Weighing up the two services I couldn't say that one is better than the other, both have advantages and disadvantages, the only way to get a decent overall picture is to use both. for the benefits of feedback to both of these great services though, here is a general comparison:

note: none of these figures are from exact tests, they are from extensive developer usage of both services as I've used them both since they were made public.

Zemanta is generally 2x as fast for average texts (the size of this post for instance) and as much as 5x as fast for longer texts. Average for Zemanta being 0.7 to 2 seconds. Average for OpenCalais being 1.5 to 10 seconds. It may also be worth noting that the availability of Zemanta is somewhat higher than that of OpenCalais, perhaps 1 in 250 calls to OpenCalais will fail.

OpenCalais does a lot more heavy work than Zemanta though, and *really* semantically analyzes the text to figure out a wealth of information. In this respect the tables are completely turned and Zemanta consitently deals with providing a few high quality known tags; where as OpenCalais often provides at least 10x as much information about a given text, including relevance and type as mentioned before. OpenCalais also extracts Facts / Events, and further it can figure out that "Jim" is also "Jim Bob", and that Jim said X about Y on date D.

Generally you can trust the data from Zemanta 99% as it deals with "known" things, however due to this in some cases very new topics (such as IPad for the first few days after its announcement) remain unknown. Due to the nature of OpenCalais and it's dealing with the unknown you need to take more time to verify what it has returned, however when OpenCalais assigns a LinkedData identifier to something or provides more information you can 99.99% trust that it is entirely accurate.

It's worth noting that both of these services do different things though, and both do it extremely well, Zemanta "tags" and OpenCalais "semantically extracts information", in some respects I was hesitant about comparing the two, as in the context of what I'm doing both are needed and both are equal, however in different contexts both do different jobs and there is a need for people to select one over the other.

Out of all the competition though, I would highly recommend both Zemanta and OpenCalais over their respective competitors, and do hope that neither of these great services ever decide to target each others markets. (e.g. they compliment each other well and both do so well because they stick to what they are good at).

extractor.data.fm details

This demo deals primarily with figuring out what a document is about; in that it aims to provide back a list of:

• Categories
  A list of 1-5 dbpedia (and therefore wikipedia) categories which the provided document would be categorized under if it were a wikipedia article and had been categorized by a huma who was knowledgeable in the subject domain(s) of the text.
• General Topics
  A short list of the general and broad-strokes Subjects covered by the document, these can are distinct from the primary specific subjects covered and the categories, and in many ways can be seen as the most common intersections between the primary specific subjects discussed.
• Primary Subjects
  These are the specific subjects covered in the document, not just the things mentioned, but the things actively discussed within the document, the primary subject matter as it were.
• Related or Mentioned Subjects
  Whilst I've termed them "related" as in dcterms:related, these are simply things which have been detected in the document or text and which are not primary subjects; in many ways "mentions" may be a more appropriate term.

Out of the above list, the two services do the heavy lifting to give the demo it's Primary Subjects and Related Subjects; in short OpenCalais' SocialTags and Zemanta's Tags give us back our Primary Subjects. Whilst OpenCalais by way of the semantic extraction provide us with the Related Subjects, namely all those extracted semantics which have the Type of a real thing (not an IndustryTerm or Event) and which are not all ready a Primary Subject; additionally those extracted semantics which are not tags but have a relevance higher than a certain score are boosted up to be Primary Subjects too.

A primary and initial function of the demo is to associate the tags returned by both services together, and figure out when each is talking about the same thing; this is covered first by dealing with the linked data they return; where both services are talking about the same thing you simply know this unambiguously due to the nature of http URIs and them both being the "sameAs" each other. After this two chunks of unhandled data remain, Zemanta tags which have not determined to be the sameas OpenCalais ones; and OpenCalais semantics which we have a string literal name for and a type.

In step Openlink Virtuoso 6.1 (open source edition!) with most of dbpedia 3.4 loaded in to do the heavy lifting from here on; Virtuoso is a really powerful bit of kit and has replaced mysql/sql server/postgres, rdf store and web dav server in my typical server stack. The public lod and dbpedia endpoints really do no justice as to just how powerful and fast Virtuoso is, queries which take a few seconds on the public endpoint return in hundredths of a second on my local (low spec) server, and the comparative performance to the aforementioned RDBMS solutions is not to be sniffed at.

To handle the typed string literals from OpenCalais, I built a custom dbpedia lookup service (using sparql over the aforementioned Virtuoso + dbpedia setup) which tries to unambiguously determine the identifier for a string literal, if it is known; the results are pretty good and I'd safely say that it gets it right in 98% of cases. This essentially turns the remaining unknown string literals in to known Linked Data URIs, and as a side benefit gives the correct full Name for the thing identified along with the correct casing and obviously much more linked data.

Remaining now the demo has a few OpenCalais semantics which are still unknown, but we know the Type and have a name for the thing; and as URIs are given to things that can be Named, I simply mint my own uri's for these and specify the OpenCalais identifier as a "seeAlso" (to be future compatible with a time where they do associate there own hash uris through to linked data).

At this point the demo has all of the Primary Subjects and Related Subjects determined and where possible linked through to additional LinkedData and human readable web documents about the subjects.

Categorization

This is where the script comes in to it's own and really leverages virtuoso, up till this point it's all been about cleaning, validating, looking up, associating and suchlike.

Given that we now have linked data HTTP URIs for all the subjects we are dealing with, and in all Primary Subject cases we also have dbpedia.org URIs the demo can start to use some of Virtuoso's more powerful features. First point of call is to get the Category intersection of all primary subjects (including the inferred categories!) via a slightly complex transitive sparql query over the dbpedia dataset. From here the demo calculates a set of primary categories which the text is related to, then it finds the general category intersection (again including inferred categories) between the primary categories, and the primary subjects. with the results returned is a wealth of numerical information which the demo dually considers and can then infer which are the General Subjects and the Categories for the text.

At some point I'll cover this part of the script in more details and give some virtuoso specific transitive SPARQL queries for you to use in your own such creations, but for now the above will have to do.

Conclusion

This extractor demo is something I've been working on and trying to achieve for about 5 years, and whilst it is still early days it's the first time the technologies have been available to both make it possible, and to utilize the results correctly to achieve what I'm aiming for overall.

The overall goal is to create a system which allows users to simply drop in content, and the system "files" it in the correct categories, lists it under the correct subjects and interlinks it with other resource via typed links such as "related resources" and looser resource lists of "also mentioned here", further benefits of such a system are that you can accurately figure out what readers are interested in and promote new content to them, you can give users the option of content streams where they can watch specific subjects or combination of subjects to be notified of their "ideal" reading. On the flip side you can also identify users and contributers interests and expertise, and correlate these together (with geo-location) to suggest others users who they may wish to collaborate with, other organisations doing the same work in the same fields and many such uses. In reality I have much of this implemented in a site I've been working on for the last year, which is just being rolled out again, and the system works extremely well with huge benefits to all involved, the site you see deals with climate adaptation and both provides a service to the general adaptation community where they can share and find knowledge, and more importantly serves organisations working on critical issues by letting them see which people / organisations / projects are doing what, where and allows them to both co-ordinate efforts and perhaps more importantly not duplicate efforts and waste resources where it counts most. This has a positive impact on the worlds poorest nations and those suffering people who these organisations are trying to work with and help.

Back to the demo, and with the context described, the extractor.data.fm demo is a quick UI around an API which is in many ways the backbone of the aforementioned system. The API is used in a semi-automated way, where the data returned by it is verified in a UI by the content author / admins who remove any unambiguous data and then hit save, from there everything is automated again and the system functions as above.

I'm unsure whether this kind of system will ever be able to be fully automated (or whether its wise to allow this) as certain scenarios just can't be covered yet, a real life example of this is an initiative called "TEA", ambiguity at this level, and with entities which are unknown to systems or even the web of data, will always be an issue at some point, as things progress it may be they are only ambiguous once, on their first discovery, but that is still once; hence why this may always have to be a semi-automated process.

The Standard Way
This way essentially exports all SSL data, certs, client and server side if you read the notes has performance penalty.

SSLVerifyClient optional_no_ca
SSLVerifyDepth 1
SSLOptions +StdEnvVars
SSLOptions +ExportCertData


The Lighter Way
This way simply passes in the SSL_CLIENT_CERT in to the env REMOTE_USER and skips the rest which you don't use (for FOAF+SSL).

SSLVerifyClient optional_no_ca
SSLVerifyDepth 1
SSLUserName SSL_CLIENT_CERT


Tested and works very nicely (again, imho).

note: Enabling SSLOptions +FakeBasicAuth will overwrite this with the Subject from the client side certificate.

GUO Graph Diff is a prototype script for performing "diffs" on RDF Graphs - not text based, rather it's graph / trple based as discussed in the DesignIssue. An implementation of patch/diff for RDF.

GUO - Graph Update Ontology pretty self explanatory, an intentionally skinny ontology aimed at enabling lightweight RDF graph updates and graph synchronisation on a per triple level.

Next up is GUO Patch in the very near future; and hopefully a full demo / implementation using HTTP Patch and such like.

More soon, all info available on the respective pages above. Any feedback or comments most welcome!

Regards,