a) The web is currently converging around web applications and mobile devices, a lot of focus is being placed on sensor networks, internet of things, and augmented reality to display information. Simply, how can these applications make use of published data readily from multiple sources if that data is not in a uniform standard?

b) The core web which people use on a daily basis is ever more silo focussed, and the size of those silo's is ever increasing - the social sector is a great example of this, and whilst there are core movements to create a more federated and distributed social web, a key blockage in the way is a lack of uniform data, often new formats are being developed, or poorly modelled application (rather than domain) specific models are making it out on to the web, and interoperability is several times harder than it could be, given the presence of uniform data. This has significant social and economic repercussions.

c) Time, a significant amount of time is invested daily by thousands (if not millions) in to re-solving the same old problems, creating a schema for this, a model for that, learning the same lessons countless people have learned before them, often the learning curve spans several years. A standard way to publish and share reusable model specific schemas (/not/ format specific like XML schema and JSON schema) would save vast amounts of developer time per annum. In addition to having significant economic impacts this would also lend to far more innovation (since more time free to innovate!) within an already important and innovative sector.

Why not "plain" RDF?

RDF has failed to be understood, adopted or loved by the general masses of the web, even many who use RDF often do not fully understand it and have many issues. Adoption has been... let's just say not good.

There are 3196 APIs on ProgrammableWeb, out of those:

• 2152 produce XML
• 1255 produce JSON
• 36 produce RDF

Perhaps more indicative though, is that those 36 are spread over 6 years, with only 1 updated so far this year, meanwhile there have been 58 new JSON based APIs in the last month alone.

Over on stack overflow, there have been 1,569,512 questions asked, 273 that's 0.017% of them, are RDF related.

The numbers are pretty clear, for all RDF's merits, and the countless benefits of the uniformity of RDF, it's just not being adopted.

To use RDF correctly requires RDF tooling, and not just tooling to parse the data (like JSON, and common usage of XML), but to use the data, to handle triples and graphs and queries, all of which requires significant investment in skills, time, and deployable technologies.

Further more, RDF data published using multiple different ontologies is difficult for people to use, the infrastructure and tooling simply doesn't exist to follow ones nose around the web and make practical use of several thousand different ontologies, that level of understanding is a good generation away, and for now all it does is serve as a blockage to adoption, and primarily as a blockage to people actually using or presenting the data. Time and time again we have seen a rallying around core ontologies, with successful mixing and matching happening more at the ontology level, than the data level. For now applications will be looking for mentions of Classes and Properties they "understand" (have a hard coded usage for).

Additionally, these difficulties in usage have lead to a second layer of centralization on the web, one which was borne from RDF, and rather ironically many of the architectural benefits of uniformity and universality are being lost. That is SPARQL, we are seeing a huge increase in SPARQL enabled datastores on the web, each of which holds a specific set of data, and each of which has key resource limitations. Practically this means that:
- clients are tightly coupled to servers
- all processing and storage weight is being handled by the servers
- data on the wire is non uniform
- clients are not using the web of data, rather they are using a datasource on the web, a datasilo.
This is a pattern which is not optimized for anybody, servers, clients, developers, data, the web, the network.

The core benefits of a web of linked data have not realized, RDF has failed to deliver them, primarily due to complexity and tooling requirements. SPARQL (positioned on the server/silo) is only compounding matters. That's not to say it cannot deliver them, or that these technologies are bad, only that they have not delivered the core benefits, yet.

Perhaps another way to put it, is that if you break things like RDBMS and Classes and Objects down you can get to triples of some sort (EAV, RDF, or to atomic relations / predicate based logic), and RDF did just this, however it was done in such a way that the data format (RDF) required a full new stack of technologies to /use/ the data, rather than being a uniform data format acting as a bridge between say classes and objects and RDBMS, a webized data model; that is to say, you can't really use "it" (RDF, the model people don't really speak of) with 95% of the deployed technology out there, you can provide an RDF view of the data from that technology, map it to RDF, but you cannot easily pull it back in and use it, and unusable data, isn't much use. There are many shades of grey between, but it's certainly more at the unusable end of the spectrum.

What can we do?

If we look at what people already do, a large proportion of web developers (most) continue to publish data via web services as XML and JSON, the common process is simple, create a schema, document it somewhere out of band (perhaps call it API documentation), publish data using that schema in some arbitrary way as XML and JSON. On the client side the same process continues, find a new API, get an XML or JSON parser, map the data as described by the API to some classes and start using it. All of this is needless work, they are showing us what works, what they can do, and how they can work with data easily. Tersely, they are missing the benefits of Uniform Data.

We can bring the benefits of uniform data to the current web 2.0, class and objects, rdbms, xml and json focussed web.

We can not only address these core issues, and bring the benefits of linked data and the semantic web to the general developer population, but we can also:
- ensure it's RDF and traditional semantic web compatible (giving "us" mountains of useful every-day data)
- provide that clear migration path to the "full" semantic web that's missing now.
- increase semantic web adoption exponentially, bringing big benefits without the high cost.

Approaches

There are two key approaches I can personally see to this:

1. Webize Classes and Objects (Java style POJOs, Data Objects, subset of UML)
2. Provide a Classes and Objects view over RDF

The first of these approaches - providing an abstract syntax for classes and objects and then defining mappings for that to XML and JSON - would bring the benefits of OWL 2 and XSD to schemas, and the benefits of "linked data" to both the schemas (/class blueprints) and instance data. It would allow data validation rules to be augmented on from sources external to the schema, it could be codified in libraries across multiple languages, it could also serve as a translation layer between Classes and Objects, NoSQL, and RDBMS, and other formats such as CSV. Additionally it would lend each schema openly being mapped to vendor specific databases, as well as vendor neutral schemas such as ANSI SQL. Furthermore, it would also lend to innovation in each layer, for example standardized queries for each kind of data could be created, with translations of those to each specific vendor or to well defined standardized languages, and even codified to work in memory in libraries (for example within instance methods or to run on GPU enabled hardware and languages). Many benefits could come from webizing what the masses already do. Other examples include providing an opportunity to refine the core datatypes on the web in a serialization agnostic way (think xsd types merged with webidl types), ensuring the correct entailments for equality are baked in to the core, providing first level support for things like lists and sets, providing a foundation upon which diff, patch, versioning can all be accomplished, providing canonicalized forms so that encryption and a data signing can be accomplished... and more I'm sure.

The second of these approaches has less wide scale benefits, but would provide a more usable abstraction layer on top of RDF, which is currently (dare I say painfully) missing. This would ultimately make working with data more familiar, a codified example could be:

var person = new Class('foaf:Person');      // external class definitions loaded from the web
person.load('http://example.org/bob#me');   // instance data loaded
print(person.name);                         // simple access to pre-known properties
person.validate();                          // in built validation from OWL 2
                                            // and XSD data type restrictions

// work with a schema class at a time..

var man = new Class('gender:Male');         // different class for different data
man.load('http://example.org/bob#me');      // same data
print(man.wife);                            // different, domain specific properties
man.expand();                               // full entailment regimes support to get
                                            // the most from schema definitions 

The best approach will become clear as time progresses, for now I'm keen and happy to work on either or both.

Just some musings..

A JSON compatible serialization of RDF

JSON is a lightweight text-based open standard designed for human-readable data interchange.

That's not what we need.

We need a lightweight text-based JSON-compatible open standard designed for machine optimized RDF data interchange.

We need it to be JSON compatible because JSON tooling is widely implemented, JSON is well supported, embraced by most web development communities, and of course it's fast and simple.

Stress: It won't be JSON, it'll be JSON-compatible, and to avoid any confusion or indicating that it may be human friendly, we probably would be wise not to put the term "json" in it's name, although we will need to put it in the media type identifier "+json".

We need to standardize because there are numerous competing and differing JSON serializations at the minute, so when somebody asks for a JSON serialization on the web, they get back different results in different places - that's unexpected behaviour, and it's not interoperability - so we need to standardize a JSON-compatible serialization of RDF, with an IANA registered mediatype.

The serialization needs to be machine optimized and usable in multiple different scenarios, else there's no point standardizing it, because other JSON compatible serializations will still be needed and therefore created. Thus we need to define the various use-cases and create a JSON-compatible serialization which caters for them.

An RDF API

There's been a lot of call for cool RDF tooling, specifically in javascript, mentions of a jquery extension and a jquery-like library - I agree that would be nice, but also say why just one? I like and want choice, freedom and to see innovation in the marketplace, I don't just use jquery, I use more libraries and modules than I could name - however..

A library is not an API, and that's not what we need to standardize.

We need to enable interoperability between libraries, and where multiple libraries may or do have overlapping and conflicting functionality, we need to align and standardize it, but only when it produces unexpected behaviour or limits interoperability.

Primarily, this means standardizing interfaces for the RDF Concepts (literals, references, triple and graph) - as soon as we do that, all libraries which implement these interfaces will immediately become interoperable. Web Developers will be able to throw data between libraries, cherry picking the features they want or the coding styles they prefer, library implementers will be able to focus on areas they specialize in, others will be able to stun us with innovative new approaches and tools we'd never have thought of ourselves, and we'll have an open and ever growing truly cool suite of interoperable classes, libraries and tools.

Next up we can standardize interfaces for things like RDF Parsers, because as soon as we do that, all libraries on each specific platform can share parsers, and these can be developed, maintained and improved independent of any specific library - it will also allow the same libraries to be used with specialist low memory parsers optimized for mobile devices, thus broadening their potential user base and giving the end users + developers the best experience possible.

Another set of interfaces which need standardized, are those which are to be implemented by multiple different vendors on the same platform, i.e. the web browsers, i.e. anything near the DOM or HTML, i.e. the RDFa API - which is already being worked on.

Finally, an interface which will make linked data and RDF easier to work with by developers (carrying on from my previous post) is a "Data Object" interface, that is a simple object which holds the description of a single subject, where the properties are keys and the values are native values. I was wrong in my previous post, serving up simple JSON objects will help, but isn't perfect, because we don't know and can't determine whether people will want to use URIs, CURIEs or terms to access properties, thus an interface will be needed to handle this.

This isn't an exhaustive list, and each paragraph above can be considered to handle a different layer in the stack, but standardizing anything substantially more than that will stifle innovation and be counter productive, people need and want different libraries with different coding styles, features and different ways to access and query data. Different tools for different jobs, and for different people doing the same job.

If you disagree, please do share and let me know!

Clarification

Earlier this week I was quoted as follows:

Nathan’s recent post on Opening Linked Data, which is worth reading in its entirety, captures the essence of the issue:

You can’t shoe horn RDF in to JSON, no matter how hard you try - well, you can, but you loose all the benefits of JSON in the first place, because the data is RDF, triples and not objects, rdf nodes and not simple values

In other words, using JSON as the basis for an RDF syntax doesn’t actually win you anything in terms of the ease of processing of that RDF. In fact, I’ll go further and say it has exactly the same bad qualities as RDF/XML.

However I have to point out that I probably didn't make the context clear, I was referring specifically to making a human friendly/optimized JSON serialization of RDF, you can however, very easily, create a machine optimized JSON-compatible serialization of RDF, without the drawbacks of XML (you just don't pin it as being human friendly JSON as outlined above) because unlike XML which requires a full XML stack, and RDF/XML which can be serialized in any one of a billion ways, we have a chance here to make a clean lightweight unambiguous serialization of RDF, one which is based on a lightweight data interchange format and not on a heavy extensible document interchange format.

Hope that clarifies :)

Why? well the sem web community is packed full of the most technically skilled and decent people I've come across so far, so it can't be that, the tooling is pretty damn good, there's loads of data, most of the data's of a high quality, wanted by developers, and certainly more than usable. The concepts, theory and technical aspects are all solid as a rock. In short it's all good, apart from one rather important detail.

Our Linked Open Data isn't really open data, not in the eyes of the common web developer at least. To most web developers on the planet, open data is something they can get access to and use easily.

A good example of 'open data' for most developers, is the Twitter API.

Here's how a developer accesses it in PHP:

  $uri = 'http://api.twitter.com/1/statuses/show/7907258268647424.json';
  $tweet = json_decode(file_get_contents($uri));
  echo $tweet->user->description;

Here's how they access it in Javascript:

  uri = 'http://api.twitter.com/1/statuses/show/7907258268647424.json';
  $.getJSON(uri, function(tweet) {
    write( tweet.user.description );
  });

The reality of the matter is that you can't do this with Linked Open Data, and that's because you can't do it with RDF - and really, honestly, if it's not that simple, the masses won't use it, because, if it's not that simple, they can't use it.

The problems with the RDF formats

They're not perfect, and they are a very mixed bunch, for a change, let's look at the negatives.

RDF/XML
- Requires full XML tooling
- Can't read or write by hand
- butt ugly

Let's be honest, unless you have a full XML and RDF stack and you know what you're doing, RDF/XML is simply a no go zone.

RDFa
- Requires HTML/DOM/XML tooling
- An extension to a markup language, designed to augment annotated documents.

RDFa is great, but not for the general data use case, it's not a simple data interchange format like JSON, and you can't publish or consume it without specialist tooling, in fact it requires an even bigger, more complicated, stack than RDF/XML.

Turtle
- Requires a custom parser
- It's not yet seen as a data format by the masses.
- Doesn't have a registered media type.

If we're all honest, Turtle is the king of RDF serializations, it's small, powerful, easy to write and read, requires minimal tooling to parse. In fact, I'd quite happily say that Turtle is the best all round data format, period.

RDF/JSON and JSON-LD
- Square peg, round hole

You can't shoe horn RDF in to JSON, no matter how hard you try - well, you can, but you loose all the benefits of JSON in the first place, because the data is RDF, triples and not objects, rdf nodes and not simple values - I fear I'd better explain, quickly:

The benefit of JSON is that you can do the following:

  var u = tweet.user;                                // nested, simple objects
  write(u.message);                                  // simply "a string of text"
  if(u.geo_enabled) {                                // a boolean true
    var d = u.statuses_count * u.favourites_count;   // numbers..
  }

Anything more complicated than that and you've lost 95% of the benefits of JSON.

Here's the code to do that if(u.geo_enabled) with JSON/RDF:

  var tweet = rdf['http://example.org/tweet/12343'];
  var user = rdf[tweet['http://example.org/property/userid']];
  var geoenabled = user['http://example.org/property/geo_enabled'];
  if( Boolean(geoenabled.value) ) {

and JSON-LD:

  var tweet, user;
  for(o in jsonld) {
    if(jsonld[o]["@"] == 'http://example.org/tweet/12343') tweet = jsonld[o];
  }
  for(o in jsonld) {
    if(jsonld[o]["@"] == tweet['twit:userid']) user = jsonld[o];
  }
  if(user['twit:geo_enabled']) {

Remember, those two examples are only for the simple if line from the benefits of JSON example, can you even imagine all four lines?

Clearly, RDF in JSON is of little to no use to anybody, you can see plainly yourself, 95% of the benefits are lost and it's just another RDF serialization that's pretty much unusable without tooling. The only benefit JSON serializations of RDF have, are that you don't require an XML stack, which is quite a large benefit tbh.

The problem with RDF

I've been a little unfair there, you see the problem isn't with the serializations, we can't make a "better" serialization of RDF for general web developers, because the real problem is that the data's RDF, it's triples not simple objects, URIs rather than simple terms, RDF Nodes with a language or type, and not just simple values.

An array of RDF triples, or a structure of RDF, just simply isn't usable in most (all?) programming languages, by a typical web developer, without specialist tooling and libraries or APIs.

Am I saying RDF is bad? no of course not, it's awesomely brilliant in every way, it powers a paradigm shift and will have huge positive effects on the web and the human race. You know the score :)

What I am saying, is that we're not backwards compatible, we're not making our data open in formats which are usable by normal developers, developers who need and want the data, want the links, but not the semwebbery. Hell even most of us who are heavy sem web users only consider the ontology+reasoning side enabled by properties-with-uris some of the time, most of the time <http://xmlns.com/foaf/0.1/name> might as well just be "name".

Opening Linked Data

So, here's what we need to do, we need to just accept that although we publish linked data as RDF, we also need to publish the data as simple objects so the world can use the data.

Given the linked data:

  @prefix : <http://webr3.org/nathan#> .
  @prefix foaf: <http://xmlns.com/foaf/0.1/> .
  @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

  :me a foaf:Person;
    foaf:age 29;
    foaf:holdsAccount [ foaf:accountName "webr3";
        foaf:homepage <http://twitter.com/webr3>;
        rdfs:label "Nathan's twitter account"@en ];
    foaf:homepage <http://webr3.org>;
    foaf:knows <http://example.com/bob#me>;
    foaf:name "Nathan";
    foaf:nick "webr3", "nath" .

  <http://example.com/bob#me> a foaf:Person;
    foaf:name "Bob" .

We also need to publish it like this:

{
  "http://webr3.org/nathan#me": {
    "a": "http://xmlns.com/foaf/0.1/Person",
    "age": 29,
    "holdsAccount": {
      "accountName": "webr3",
      "homepage": "http://twitter.com/webr3",
      "label": "Nathan's twitter account"
    },
    "homepage": "http://webr3.org",
    "knows": "http://example.com/bob#me",
    "name": "Nathan",
    "nick": [ "webr3", "nath" ]
  },
  "http://example.com/bob#me": {
    "a": "http://xmlns.com/foaf/0.1/Person",
    "name": "Bob"
  }
}

and now one can do: me.holdsAccount.label and get back a string, in any language.

What have we lost?
Well nothing in the grand scheme of things because we're still publishing the RDF in other formats via conneg, however in this specific serialization of the data: we've lost the properties, the .language and the .type, although basic types are still there, dates are detectable, numbers are supported, booleans are supported, everything else is just a PlainLiteral, a string.

What's still there?
The data, http names for things, follow your nose, rdf types ++usable-accessible-data in a web standard format. It's 3.5 if not 4.5 star data!

Other considerations
Probably be wise to allow direct access to a .json URI too, so people can simple-GET the data (in addition to exposing via conneg).

Should be an easy hit, any good reasons why not?

The other RDF Serializations

While we're here, there are a few other things that need tidied up, properly.

Turtle
Let's make it the standard RDF serialization, with a proper registered media types of application/turtle and text/turtle, fix any bugs in the spec (if any), and possibly allow an optional comma in those lists (1,2,3).

Let's just accept that RDFa is great, but sometimes you just want to embed a chunk of RDF in an HTML document, you know we all want to on occasion, people are shouting for it, it's easy to do, to deploy, doesn't break anything, and it's a really easy hit - so, in the Turtle standard spec pop a note that shows how to include it in an HTML document.

<script type="text/turtle">
  // turtle in here, as-is, no special encoding
</script>

RDF/XML
Leave it as is, let it run it's course naturally, and in many respects just forget it moving forwards, everybody supports it already, that won't change, but there's approximately zero need to keep pushing it.

RDFa
As awesome as it is, just see it for what it is, RDF in attributes, it's like the missing markup features of HTML, for annotating documents and describing the things described in the documents with annotations. It's not a simple data or RDF format, and it's not really suited to just dropping chunks of machine readable RDF in to an HTML document, Turtle in HTML will do that far better.

RDF/JSON and JSON-LD
Standardize a JSON serialization as a replacement / alternative to XML - but, admit and stipulate before hand that it's not usable as-is, or really writable and is arguably human readable. It simply needs to be a fast, unambiguous, optimized for the machine, RDF serialization - no bells and whistles for humans, no "12^^xsd:type" in a string - just something that you can JSON.parse and run circa 10-20 standardized lines of code over to get back an RDFGraph of RDFTriples.

Fin & end.

This is quite a dry subject, and I could be wrong (in fact in some ways I want to be proved wrong, this is how we learn), but do read on if you're interested.

Example

Over the past few months I've hit on a number of occasions where I was convinced I needed Named Graphs in order to address the task at hand.

A notable example is the scenario where using WebAccessControl and the ACL ontology, a system would have to figure out just who should be given access to a resource, and who should be denied.

In this example I'll cover the notion of ACL for "groups" in a linked data world.

The task at hand is to allow access if:
the graph serialized within the document obtained by dereferencing the URI of the group states the <webid#me> is a member.

Otherwise written as:
if we dereference <groups#admin> does the graph returned include the following { <groups#admin> sioc:has_member <webid#me> }

Or in SPARQL:

ASK
GRAPH <groups> {
  <groups#admin> sioc:has_member <webid#me>
}

In this example we *do not* want to dereference the users webid to see if the graph returned specifies that { <webid#me> sioc:member_of <groups#admin> } , or indeed consider the open world possibilities that another yet unknown graph could assert that the user is a member of our admin group, as that would breach security.

The ACL

To proceed with the example, consider the following ACL:

[] a acl:Authorization ;
	acl:accessTo <https://example.org/sensitive> ;
 	acl:agentClass :mygroup ;
 	acl:mode acl:Read .

:mygroup owl:equivalentClass [
 	a owl:Restriction ;
 	owl:hasValue <groups#admin> ;
 	owl:onProperty [ owl:inverseOf sioc:has_member ];
 	] .

The Problem

The problem proposed by this ACL is that any of the following four sets of triples would infer that <webid#me> would qualify as an instance of :mygroup (or a member of <groups#admin> if you prefer).

• <webid#me> sioc:member_of <groups#admin> .

• <webid#me> _:x <groups#admin> .
  _:x owl:inverseOf sioc:has_member .

• <groups#admin> sioc:has_member <webid#me> .

• <groups#admin> _:y <webid#me> .
  _:y owl:inverseOf sioc:member_of .

In other words, the ACL does not specify a "Named Graph" to query, and at the moment, no way exists to specify with (OWL or RDF) which "Named Graph" to query / trust.

This, point in case, is one example where I saw the need for Named Graphs in RDF and OWL.

Another way of looking at it

You will have noticed the notion of "Named Graphs" creeping in above, seems like a logical thing to say, especially when you consider that to process this ACL and grant access you'd probably use SPARQL, and specify a Named Graph to query over. However, much of what follows arose because I'd decided not to use SPARQL, and rather to code an ACL processor in my preferred language.

If you consider the situation, the ACL processor which decides if access should be granted or not, must implicitly "trust" the document which contains the serialized ACL graph. That is to say, that it must by extension trust any resources pointed to by said ACL, and if it doesn't then the ACL isn't fit for the purpose.

It's also important to note that "trust" is context specific, in this case we trust the resources pointed to by the ACL for the purpose of WebAccessControl.

One could then pretty quickly conclude that in this scenario the ACL processor already know's how to process the ACL, it must only use resources it trusts, therefore it must only allow access if the graph serialized within the document obtained by dereferencing the URI of the group states the <webid#me> is a member.

(because <groups#admin> is specified in the ACL, and thus by extension, trusted)

Named Graphs in SPARQL

The aforementioned logic would also apply if I was using SPARQL to process the ACL, it would equate to the ACL processor asking:

ASK
GRAPH <groups> {
  <groups#admin> sioc:has_member <webid#me>
}

But again this is very context specific to the example, let's consider for a moment that the URI for the group could have been a non-fragment URI, <groups/admin> for example.

This leads us to an important problem, when we dereference <groups/admin> it would have to 303 See Other through to a different URI, let's say <data/groups/admin> - which would then mean that the Named Graph to be used was <data/groups/admin> - this URI, you may note, we do not know when we are writing our ACL; so if we ASKed the above SPARQL, the results would always come back negative, since their is no GRAPH <groups>.

The URI of the Named Graph issue is compounded by modern web servers and publishing practises, because <data/groups/admin> could easily be content negotiated (or rewritten), thus giving various final URI's of <data/groups/admin> or <data/groups/admin.rdf> or <data/groups/admin.ttl> or <data/groups/admin.n3> and so forth. One could quite easily (and often does) end up with the same Graph repeated multiple times within a quad store, all under "different" "Named Graphs".

I'll expand on a possible way of addressing this problem further on.

Directionality

Previously I mentioned that the ACL processor didn't have a problem with the above ACL, because it by nature trusted all resources which were mentioned in the ACL graph. However, again this is very context specific.

Let's consider for a moment an inverted ACL, where we want to allow access if:
the graph serialized within the document obtained by dereferencing the URI of the users webid states that <webid#me> is a sioc:member_of <groups#admin>.

We don't know the users webid ahead of time when we write the ACL, so again we have no way of writing how to trust a resource - it is critical to note that even if RDF(2) did support the concept of Named Graphs, it still wouldn't address the situation because we wouldn't know the Named Graph ahead of time, in order to trust it!

If we now consider the following ACL:

[] a acl:Authorization ;
	acl:accessTo <https://example.org/sensitive> ;
 	acl:agentClass :mygroup ;
 	acl:mode acl:Read .

:mygroup owl:equivalentClass [
 	a owl:Restriction ;
 	owl:hasValue <groups#admin> ;
 	owl:onProperty sioc:member_of;
 	] .

The outcome of our previous logic concludes that again we should be querying the "trusted" resource <groups#admin>, which gives us another problem, that's not the resource we want to be asking in this scenario.

The only thing that remains, and I'll later argue the only thing that ever matters in a web of linked data, is direction.

If we analyse the first ACL closer, we can see that we ultimately used the direction inferred by the presence of owl:inverseOf to place <groups#admin> in the subject position, rather than the value/object position it could have been in, indicated by the presence of owl:hasValue. (bare with me).

In this example, we can use the strong semantics of owl:hasValue (and lack of owl:inverseOf) to place <groups#admin> in the value/object position, and thus our ACL processor can come to the outcome we want, which is to look for the a triple with the meaning { <webid#access> sioc:member_of <groups#admin> }, and that means dereferencing the URI in the subject position, in other words asking the graph serialized in the document returned by GETting <webid> if it contains such a triple.

I've applied some understanding to OWL that quite simply isn't there though, as I earlier stated both ACL examples could easily equate to looking for any one of those four sets of triples.

However, this is the point - machine understanding of data is in the domain of the machine, the application doing the processing. And "truth" or "trust" is entirely context specific.

I'm increasingly convinced that the combined context of the data in a graph and the context under which that graph is being queried, specifies or infers in which direction you want to be reading, and directionality can be determined with linked data by dereferencing whichever uri you place on the left / in the subject position.

I recently found that Tim Berners-Lee wrote about this in a blog post entitled Backward and Forward links in RDF just as important:

One meme of RDF ethos is that the direction one choses for a given property is arbitrary: it doesn't matter whether one defines "parent" or "child"; "employee" or "employer". This philosophy (from the Enquire design of 1980) is that one should not favor one way over another. One day, you may be interested in following the link one way, another day, or somene else, the other way.

Key here is the sentence "One day, you may be interested in following the link one way, another day, or somene else, the other way.", and that is exactly what all these examples are doing, following a link one way, or the other way.

To conclude this part, in every scenario thus far where I've thought I needed Named Graphs, it turns out that I in-fact needed directionality - and because I'm dealing with Linked Data, whatever I place in the subject position defines the URI which I need to dereference, and ultimately the Graph(s) which are considered when resolving the answer to the question being ASKed.

I'd thus suggest that "Named Graphs", do not exist in a web of data, they are needed in N3 and when using rules, because all data is often in a single file, however that is not the case for Linked Data, where we dereference.

Back to SPARQL and Named Graphs

Previously I mentioned the complications with the way we currently use named graphs in SPARQL and in our quad stores, where the URI we end up using could literally be, anything; and often we get duplicate data under different graphs.

To address this, I'd suggest that what we should be storing as the graph ?g value, is not some made up "named graph" but rather: the dereferenced URI which we initially requested.

• in the case of <group#admins> this would be <group>.
• in the case of <group/admins> this would be <group/admins>

To clarify, *never* the URI that a GET request finally resolves to, and *always* the initial dereferenced URI we requested.

The above ensure that we'd never have duplicate data in our quad stores again, that SPARQL queries including a FROM clause always dereferenced, that publishers and web server administrators were free to relocate and restructure their data, and ultimately make for a much nicer, healthier web of data.

Cool URIs don't change, and they wouldn't, just because the final document serializing a graph may move to a different URI, doesn't mean the original URI has to change.

Conclusion

Apologies for the length of the post, but I figured everything needed covered, in context. Simply put we need to focus less on Named Graphs (which IMHO aren't needed) and focus more on directionality. Every problem I've encountered thus far is covered by what Tim said years ago: "One day, you may be interested in following the link one way, another day, or somene else, the other way."

Comments?

I'm primarily documenting this journey for my own benefit, for reference and to unload it from my brain; but hopefully it'll be of use to the wider community and any feedback will be massively appreciated.

Here goes, I'll start by analysing the web thus far:

The Web till now

The power and the success of the web so far, in my opinion, has come from four main things:

• The URI
• Hyperlink
• The Resource
• HTTP and it's RESTful design.

More importantly it's the combination of the four working together that makes the web so great, because, they let you cut out everything else and go straight to the resource you want. This is a point that we need to concentrate on for a minute.

Going straight to the resource

Back at the start of the web, this allowed people to (for the first time) jump from a resource in the bowels of one companies hierarchy straight to another resource in a different companies hierarchy - very much like reading a book, looking at the references and suddenly having the book or paper it references right there in your lap - amazing to say the least.

Skip forwards a few years and we have the search engines, suddenly simply by typing a few keywords you can jump right to a resource (page/image/...) anywhere on the web. Fast forward some more and we get to web 2.0 where again people are amazed every time direct access to a resource is exposed. Yup.. all your web 2.0 is just this simple principal..

An RSS feed, well it lets you read a resource (a post) outside the context of the website and inside lets say google reader. You can rip a resource (video) out of youtube and embed it anywhere you like. You can interact with a web application super fast thanks to targeting a resource directly with say ajax and only updating that resource rather than updating the entire view (page). You can interact with web applications using desktop clients because they let you access one resource at a time; and so on and so forth, virtually every improvement you see on the web comes down to that one thing, directly accessing a resource (and creating resources at a more granular level).

How we made the web faster

Negating the rather obvious upgrades in technology over the years, there is one primary thing that speeds up the web and virtually everything computer oriented, the cache.

Before all the web 2.0 stuff, resource caching was at an all time high and was making the web faster for all of us; caching at the resource level is enabled by HTTP and its RESTful design. Control Data allows us to limit how much information needs transferred over the web, request an image once and it gets transferred, request it a second time and thanks to caching and HTTP odds are very high that it won't get transferred again. When you consider that the average web page can easily have 30, 50, 300 images and static files embedded in it this is a huge speed increase, and frankly one we could not live without.

Skipping forwards to web 2.0 and the present day again, we've gone wild with caching; anybody who's been involved with a high traffic site will tell you that the only way to do it is to cache everything you can; from data in memory, through to code and op code caches. But this is only half the story.. a strange this has happened..

How we made the web slower

Simply, we forgot HTTP and a RESTful web somehow - that all important web caching whether it be at intermediate servers or in a web browser, it's forgotten.

To illustrate, if you view an image and then view it again, it'll be there instantly - why? because last-modified, etag and other control data is sent by great web servers like apache for static files, until you force a refresh or the file changes on the server you'll simply get a 304 response telling whatever cache down the line to use it's own copy instead. Now, try jumping on to a web page, even this one and you'll find the whole thing is reloaded, every time. I'd estimate that circa 80% of all pages you visit are fully reloaded every single time you see them, if not more.

Here's the reason - most pages are generated by scripts now, and something that goes unnoticed by most developers is that the web server (like apache) hand over *full* control to the language runtimes, and in turn to the developer. In other words, unless developers are calculating, receiving and sending control data for each response, and validating every http message in to their scripts, then most of the benefits of HTTP and RESTful design are completely lost; especially caching.

Here's a fact, whether you agree with it or not, to me it is a fact: the web has to be RESTful for it to work properly, whether it's a web of documents, or a web of data, or both.

Looking at the current state of Linked Data

Linked Data is amazing because it takes the big four I mentioned earlier (URI, Hyperlink, Resource, HTTP) to a new level; we create resources at the most granular level possible, assign them URIs, link them together with typed hyperlinks then expose them via HTTP.

Notice I didn't mention REST in there? that's because I (and I'm not the only one) don't feel that Linked Data is currently RESTful. And as we can learn from web 2.0, unless this is addressed we'll face major problems down the line. In addition, because of this lack of RESTful-ness I feel like the data isn't linked; simply using URIs from different datasets on both sides of a triple does not link those datasets, well not from a client perspective anyway.

To expand and refining the issues:

SPARQL Silos

Issue one, is that SPARQL and the servers with RDF stores which power it are positioned at the wrong side of the client / server relationship imho. Because each major dataset effectively has it's own server and access point (SPARQL interface) it means that when you query it, it can only return the Linked Data which it stores. This leaves us three options at the minute:

• let that server pull in remote Linked Data and store it too (which makes the server fill up and slow down, and turns it in to a silo).
• use one great big server that tries to store all the linked data (which feels like a silo all over again to me, not distributed at all).
• Run our own server and only store limited data in it (limited.. and again a silo I guess)

If we moved SPARQL to the client side however, then all it would need is a starting point from which it could traverse the web of data, only pulling in what it needed for a query. This may sound slow but if all data was exposed as resources like it should be, and with control data so it could be cached, this slow down would soon disappear; lesson from web 1 and 2!

With regards the caching, this could happen at traditional intermediate caches within the internet and at ISPs, locally in client side triples stores (like a browsers cache) or the existing big servers that attempt to store all the linked data could be repositioned as linked data caches.

For example a small RDF document could simply delegate seeAlso http://datacac.he/http://subject.uri and that linked data cache could return back all the information it knows about the subject by returning the RDF results of a SPARQL describe. This alone would be a HUGE speed up, prevent silos and create a real web of data.

In addition, this would keep all linked data transferred through the web in RDF format, and thus machine readable and typed. At the minute we have lots and lots of SPARQL queries, which essentially are just untyped junk data that a machine couldn't possibly understand - SPARQL results remove all the goodness from RDF and give us something that is domain and developer specific, not re-usable. Think about that for a moment..

Clarification: I'm not saying SPARQL + RDF stores shouldn't be on the server side, they should as they are needed in most cases, I'm simply saying that the primary interface to linked data shouldn't be SPARQL over HTTP to a remote SPARQL endpoint. Rather we should be accessing RDF documents, or entities if you like via HTTP.

RESTful RDF

Issue two, the focus has been on getting data on the web, finding ways to link it, access it, store and query these vast datasets; and the work done thus far is amazing! But now that's handled it's time to go back to basics and find ways of both getting and publishing Linked Data RESTfully, at a granular per resource level.

This means handling RDF like ATOM, and essentially making atompub all over for RDF (as many are thinking and working on). I feel that regardless of what's implemented behind the interface, and whether triples stores and SPARUL are used, we still need to manage RDF / Linked Data in terms of documents and entities for it to be RESTful.

An additional issue raised by this is loosing the notion of a quad, g s p o, Named Graphs are vastly important to linked data, but we need to get named graphs in to triples and out of quads so we are always working with RDF through HTTP.

Also worth noting that temporal, provenance, multi-language, multiple representations etc will all need handled too; without using any HTTP extensions; no point half baking it or making the solution dependant on drafts - needs to work with the Universal Interface!

First Step

The above means one of the first challenges and things I'll try to tackle, is to find a way to fit a RESTful RDF publishing and exposing protocol in to the shared http space on a web server; taking in to account things like content negotiation, multiple representations, versions / time varying representations, and backwards compatibility with the current web.

note: I'm not going to define the protocols, plenty of more intelligent people than me are working on these things, just leverage a space where a full RESTful protocol can work in unison with the way we currently do things so that it's transparent to browsers and visible to linked data clients. This should then allow a stable test environment to try out different ways of doing things and test that the current web doesn't break.

To be continued.. often and frequently. I'm blocked on my current, v important project, and need to address these things.

I've grouped all the links in to two main sections, and then sub-grouped by how they make sense in my head! :)

Web, HTTP and REST

Roy T. Fielding Dissertation - Architectural Styles and the Design of Network-based Software Architectures Of particular relevance and note are chapters 4-6 (many only ever read chapter 5 and miss the context + summary *needed* in chapters 4 and 6!)

• Chapter 4 - Designing the Web Architecture: Problems and Insights
• Chapter 5 - Representational State Transfer (REST)
• Chapter 6 - Experience and Evaluation

Roy T. Fielding - REST APIs must be hypertext-driven
Discussion on HTML5 and RESTful HTTP in browsers
Discussion on URIs Resources and Switching content types w/ REST angle (v good)

RFC 2616 HTTP/1.1 and the HTTPbis Working Group HTTP/1.1 update in parts:

1. Messaging
2. Semantics
3. Payload
4. Conditional
5. Range
6. Cache
7. Authentication

Linked Data and the Semantic Web

Linking Open Data Community Project
Linked Data Applications
Equivalence Mining and Matching Frameworks
Linked Data Browsers, Mashups and other Client Applications

Dataset Dynamics - On the Dynamics of Linked Datasets
Realizing a write-enabled Web of Data
Web Access Control (WAC) - a decentralized system for allowing different users and groups various forms of access to resources where users and groups are identified by HTTP URIs.
Discussion of the WAC vocabulary
Socially Aware Cloud Storage Design Note
Distributed Social Networking through Socially Aware Cloud Storage from TimBL
AWWSW - "Architecture of the World Wide Semantic Web" Task Force

SPARQL 1.1 Uniform HTTP Protocol for Managing RDF Graphs
A Linked Data Frontend for SPARQL Endpoints
RAP - RDF API for PHP V0.9.6
Inav the Terrible - An idea for posting RDF through HTTP.

Talis Changesets
Triplify Update Vocabulary

skos as atom

RFC 4287 - The Atom Syndication Format
RFC 5023 - The Atom Publishing Protocol
AtomPub Tombstones - The Atom "deleted-entry" Element
RFC 5005 - Feed Paging and Archiving
Versioning Link Relations - Link Relation Types for Simple Version Navigation between Web Resources

Named Graphs, Provenance and Trust
Accessing Site-Specific APIs Through Write-Wrappers From The Web of Data
Provenance Information in the Web of Data - LDOW 2009 paper
Using Reification To Extend RDF (historical reification approach)
RDF Policy-based URI Access Control for Content Authoring
The Open Provenance Model Core Specification (v1.1)
W3C Provenance Incubator Group

History of the Web 1945, 1980 through 1997 on W3
LEIRI - Legacy extended IRIs for XML resource identification The type of "URI" used in xml:base
CSHALS 2010 W3C Semanic Web Tutorial
Mindswap online RDF Converter
W3 online RDF Validator