server side js with v8

Just logging some thoughts and scoping work..

v8 and server side ecma-262 (javascript) is a fantastic thought, and definitely the future imho (universal language).

To get going and start making server side js web applications there are two good approaches, v8cgi which aims more at letting you run js programs as a cgi app (like you do with php cgi) under apache and similar, seems to be picking up speed, nice approach (workable) - the other is node.js which is off to a great start as a pure server side environment where 'HTTP is a first class protocol' which just proves what a good start Ryan Dahl has made.

Briefly to touch on v8, it allows you expose C code to javascript, which is.. well perfect, would be good to see far more lib's exposed to js using tools like cproxyv8 and v8-juice - in particular I'm looking at you librdf (and an interesting sub-thought, apache and openssl).

Back to node.js, seems to be exciting already has a strong API to get some great demo's and projects off the ground, and very well aligned with the W3C work (or maybe what-wg..), specifically on the Web Worker API front; in fact there are already implementations of Web Sockets for node.js (and socket.io) - nodelint (jslint for node) is a handy addition, whilst some projects seem to be going all in, like Persevere.

As for adapting current code to work on the server side, you'll want to be looking in to the modules .. which.. well to be honest that's a new link I just found, and not what I was thinking off.. emm, I think that provides almost everything I need - might be about to jump back in with two feet!

The rest of this post was about to mention php.js, harmony framework (Converts PHP 5 code into Javascript), and then mascara and of course haxe which compiles to js amongst many other targets.

Might have to come back to this post.. for anybody who follow what I do, thus far I'd figured out that node.js is http+tls friendly, you can get a clients certificate (for foaf+ssl) and you can send application webid+ssl certs too, although up till now I hadn't found a way to generate certificate (let alone using spcak) but that could be wrong..


Something's missing in the Web of Linked Data


Maybe we don't need Named Graphs

In this post I'll put forward an argument that perhaps the "web of linked data", and thus RDF(2)/OWL(2), doesn't need any concept of Named Graphs.

This is quite a dry subject, and I could be wrong (in fact in some ways I want to be proved wrong, this is how we learn), but do read on if you're interested.

Example

Over the past few months I've hit on a number of occasions where I was convinced I needed Named Graphs in order to address the task at hand.

A notable example is the scenario where using WebAccessControl and the ACL ontology, a system would have to figure out just who should be given access to a resource, and who should be denied.

In this example I'll cover the notion of ACL for "groups" in a linked data world.

The task at hand is to allow access if:
the graph serialized within the document obtained by dereferencing the URI of the group states the <webid#me> is a member.

Otherwise written as:
if we dereference <groups#admin> does the graph returned include the following { <groups#admin> sioc:has_member <webid#me> }

Or in SPARQL:

ASK
GRAPH <groups> {
  <groups#admin> sioc:has_member <webid#me>
}

In this example we *do not* want to dereference the users webid to see if the graph returned specifies that { <webid#me> sioc:member_of <groups#admin> } , or indeed consider the open world possibilities that another yet unknown graph could assert that the user is a member of our admin group, as that would breach security.

The ACL

To proceed with the example, consider the following ACL:

[] a acl:Authorization ;
	acl:accessTo <https://example.org/sensitive> ;
 	acl:agentClass :mygroup ;
 	acl:mode acl:Read .

:mygroup owl:equivalentClass [
 	a owl:Restriction ;
 	owl:hasValue <groups#admin> ;
 	owl:onProperty [ owl:inverseOf sioc:has_member ];
 	] .

The Problem

The problem proposed by this ACL is that any of the following four sets of triples would infer that <webid#me> would qualify as an instance of :mygroup (or a member of <groups#admin> if you prefer).

  • <webid#me> sioc:member_of <groups#admin> .
  • <webid#me> _:x <groups#admin> .
    _:x owl:inverseOf sioc:has_member .
  • <groups#admin> sioc:has_member <webid#me> .
  • <groups#admin> _:y <webid#me> .
    _:y owl:inverseOf sioc:member_of .

In other words, the ACL does not specify a "Named Graph" to query, and at the moment, no way exists to specify with (OWL or RDF) which "Named Graph" to query / trust.

This, point in case, is one example where I saw the need for Named Graphs in RDF and OWL.

Another way of looking at it

You will have noticed the notion of "Named Graphs" creeping in above, seems like a logical thing to say, especially when you consider that to process this ACL and grant access you'd probably use SPARQL, and specify a Named Graph to query over. However, much of what follows arose because I'd decided not to use SPARQL, and rather to code an ACL processor in my preferred language.

If you consider the situation, the ACL processor which decides if access should be granted or not, must implicitly "trust" the document which contains the serialized ACL graph. That is to say, that it must by extension trust any resources pointed to by said ACL, and if it doesn't then the ACL isn't fit for the purpose.

It's also important to note that "trust" is context specific, in this case we trust the resources pointed to by the ACL for the purpose of WebAccessControl.

One could then pretty quickly conclude that in this scenario the ACL processor already know's how to process the ACL, it must only use resources it trusts, therefore it must only allow access if the graph serialized within the document obtained by dereferencing the URI of the group states the <webid#me> is a member.

(because <groups#admin> is specified in the ACL, and thus by extension, trusted)

Named Graphs in SPARQL

The aforementioned logic would also apply if I was using SPARQL to process the ACL, it would equate to the ACL processor asking:

ASK
GRAPH <groups> {
  <groups#admin> sioc:has_member <webid#me>
}

But again this is very context specific to the example, let's consider for a moment that the URI for the group could have been a non-fragment URI, <groups/admin> for example.

This leads us to an important problem, when we dereference <groups/admin> it would have to 303 See Other through to a different URI, let's say <data/groups/admin> - which would then mean that the Named Graph to be used was <data/groups/admin> - this URI, you may note, we do not know when we are writing our ACL; so if we ASKed the above SPARQL, the results would always come back negative, since their is no GRAPH <groups>.

The URI of the Named Graph issue is compounded by modern web servers and publishing practises, because <data/groups/admin> could easily be content negotiated (or rewritten), thus giving various final URI's of <data/groups/admin> or <data/groups/admin.rdf> or <data/groups/admin.ttl> or <data/groups/admin.n3> and so forth. One could quite easily (and often does) end up with the same Graph repeated multiple times within a quad store, all under "different" "Named Graphs".

I'll expand on a possible way of addressing this problem further on.

Directionality

Previously I mentioned that the ACL processor didn't have a problem with the above ACL, because it by nature trusted all resources which were mentioned in the ACL graph. However, again this is very context specific.

Let's consider for a moment an inverted ACL, where we want to allow access if:
the graph serialized within the document obtained by dereferencing the URI of the users webid states that <webid#me> is a sioc:member_of <groups#admin>.

We don't know the users webid ahead of time when we write the ACL, so again we have no way of writing how to trust a resource - it is critical to note that even if RDF(2) did support the concept of Named Graphs, it still wouldn't address the situation because we wouldn't know the Named Graph ahead of time, in order to trust it!

If we now consider the following ACL:

[] a acl:Authorization ;
	acl:accessTo <https://example.org/sensitive> ;
 	acl:agentClass :mygroup ;
 	acl:mode acl:Read .

:mygroup owl:equivalentClass [
 	a owl:Restriction ;
 	owl:hasValue <groups#admin> ;
 	owl:onProperty sioc:member_of;
 	] .

The outcome of our previous logic concludes that again we should be querying the "trusted" resource <groups#admin>, which gives us another problem, that's not the resource we want to be asking in this scenario.

The only thing that remains, and I'll later argue the only thing that ever matters in a web of linked data, is direction.

If we analyse the first ACL closer, we can see that we ultimately used the direction inferred by the presence of owl:inverseOf to place <groups#admin> in the subject position, rather than the value/object position it could have been in, indicated by the presence of owl:hasValue. (bare with me).

In this example, we can use the strong semantics of owl:hasValue (and lack of owl:inverseOf) to place <groups#admin> in the value/object position, and thus our ACL processor can come to the outcome we want, which is to look for the a triple with the meaning { <webid#access> sioc:member_of <groups#admin> }, and that means dereferencing the URI in the subject position, in other words asking the graph serialized in the document returned by GETting <webid> if it contains such a triple.

I've applied some understanding to OWL that quite simply isn't there though, as I earlier stated both ACL examples could easily equate to looking for any one of those four sets of triples.

However, this is the point - machine understanding of data is in the domain of the machine, the application doing the processing. And "truth" or "trust" is entirely context specific.

I'm increasingly convinced that the combined context of the data in a graph and the context under which that graph is being queried, specifies or infers in which direction you want to be reading, and directionality can be determined with linked data by dereferencing whichever uri you place on the left / in the subject position.

I recently found that Tim Berners-Lee wrote about this in a blog post entitled Backward and Forward links in RDF just as important:

One meme of RDF ethos is that the direction one choses for a given property is arbitrary: it doesn't matter whether one defines "parent" or "child"; "employee" or "employer". This philosophy (from the Enquire design of 1980) is that one should not favor one way over another. One day, you may be interested in following the link one way, another day, or somene else, the other way.

Key here is the sentence "One day, you may be interested in following the link one way, another day, or somene else, the other way.", and that is exactly what all these examples are doing, following a link one way, or the other way.

To conclude this part, in every scenario thus far where I've thought I needed Named Graphs, it turns out that I in-fact needed directionality - and because I'm dealing with Linked Data, whatever I place in the subject position defines the URI which I need to dereference, and ultimately the Graph(s) which are considered when resolving the answer to the question being ASKed.

I'd thus suggest that "Named Graphs", do not exist in a web of data, they are needed in N3 and when using rules, because all data is often in a single file, however that is not the case for Linked Data, where we dereference.

Back to SPARQL and Named Graphs

Previously I mentioned the complications with the way we currently use named graphs in SPARQL and in our quad stores, where the URI we end up using could literally be, anything; and often we get duplicate data under different graphs.

To address this, I'd suggest that what we should be storing as the graph ?g value, is not some made up "named graph" but rather: the dereferenced URI which we initially requested.

  • in the case of <group#admins> this would be <group>.
  • in the case of <group/admins> this would be <group/admins>

To clarify, *never* the URI that a GET request finally resolves to, and *always* the initial dereferenced URI we requested.

The above ensure that we'd never have duplicate data in our quad stores again, that SPARQL queries including a FROM clause always dereferenced, that publishers and web server administrators were free to relocate and restructure their data, and ultimately make for a much nicer, healthier web of data.

Cool URIs don't change, and they wouldn't, just because the final document serializing a graph may move to a different URI, doesn't mean the original URI has to change.

Conclusion

Apologies for the length of the post, but I figured everything needed covered, in context. Simply put we need to focus less on Named Graphs (which IMHO aren't needed) and focus more on directionality. Every problem I've encountered thus far is covered by what Tim said years ago: "One day, you may be interested in following the link one way, another day, or somene else, the other way."

Comments?


Web of Data illustrated, meet Bob.

bob

Something to try

Next time you start an application, why not create a subdomain for your data tier, create a script for each SQL query and call it via HTTP (honestly you won't notice a speed difference), similarly expose all static files your system uses on the same (or a different) sub domain. Here's the advantages:

  1. You can scale without changing your application
  2. You can swap servers without changing your application
  3. You can move data around your server and take advantage of all the HTTP servers features.
  4. You can cache your resources and utilize HTTP caching.
  5. You can distribute or relocation your application anywhere on the net without worrying about data.
  6. You can migrate over to different data systems (swap database vendors etc) without changing your application.
  7. You can call every resource on the web in the same manner, from APIs through to linked data and all in between.
  8. You're critical SQL and data mapping code will be fully abstracted and in self contained files (ultra easy to edit, bug fix, maintain).
  9. You can take advantage of HTTP logging, and HTTP status codes instead of custom exceptions.
  10. You have a ready made RESTful API in to your data tier, so you can hook on other applications, or open it up to third parties, or the entire net.

So much more, all by one simple step - and that's only the first half of the cartoon ;)


Generations of the Web - an Overview.

I feel it's very important to note where we are coming from, and where we are headed with regards the web.

Generations One and Two

Historically we have been through two generations of the web so far, not web 1 and web 2.0, but rather the first round of mounting the presentation tier on the web (static publishing documents and media), then the second round of mounting the application tier on the web (from forms through to the current api's we see everywhere).

The next round, and what many are currently hacking at, is mounting the data tier on the web, mainly via the "Linked Data" movement which you may have noted.

This third round is critical for two primary reasons:

  1. By mounting the data tier on the web, it allows us to remount the application tier again, but this time properly, where HTTP is the API between the Application tier and the Data tier. (true REST)
  2. The "Internet of Things" is very much dependant on the "Things" speaking the same language and sharing a single Data Model, where the model remains the same but the vocabulary can change.

Generation Three

"Linked Data" is in many ways the solution to the above, because it has a Universal Data Model (EAV), a Universal API (HTTP), and Universal Addressing & Identification (the duality of http URIs) - more importantly though, "Linked Data" provides the means to mount and expose both ABox and TBox statements on the web - this is the key detail where RDF wins over other EAV based data models; the ability to store both the data and the vocabularies on the web, and both in the same manner.

Linked Data is currently tied (but not bound) to RDF. However, RDF still has very critical limitations, primarily it doesn't have any notion of Named Graphs or Quads. Trust and provenance are essential moving forwards (with quads you can select which data your application trusts, and which it doesn't, based on the source, the Named Graph - without it you can't), as is the ability to transfer data from multiple sources at the same time (update streams, batch operations, merged data with provenance - again needing quads).

The above means we need an RDF2 (or standardisation & adoption of N3+rules), and moreover we need standardized non-xml serializations of said RDF(+2), the most important being rdf/json.

This brings me to HTML5, the key here isn't the new HTML5 document format, the semantics, the ability to embed microdata or anything like that - it's the introduced (or implied) dependency on JS /ECMAScript via the JS APIs. This leads to JS being rolled out on to the majority of devices - it's also important to note the server side JS movement in this too.

Generation Four

We've had a "Web of Documents", we're building a "Web of Data", the next logical step is to have a "Web of Applications", for this we need two things: a Universal Programming Language (JS...) and an "Internet of Things" which support the universal data model and the universal programming language.

As far as I'm concerned HTML5 plays a critical part in the big picture:

- Short term, it at least unites the browser vendors to better support the techs which developers use.
- Presently (and from here on) it allows us to start hacking at the web of applications.
- In the future it's legacy will be it's introduction of JS as a Universal Programming Language.

Sides:
It stands to reasons that RDF/JSON (or whatever supersedes RDF serialized in JSON) will also play a major part in this round.

We may well see a shift from client-server, server-server to application-application; where each machine on both sides comprises of HTTP Client, Server and Cache.

Generation Five

Leading on from here we get to the fifth round of the web, remounting the Presentation Tier, but this time where the presentation tier speaks to the application tier through HTTP, the Universal Interface.

HTML (together with JS and CSS) again plays a critical role in round five, because up until this point it remains the only way for the hackers to create this fifth generation, notably it will be a lot more than we are have now, after years of maturity, hacking and support universally. Naturally then we can assert that HTML will remain the core of the Web's presentation tier so long as the Web exists.

Additional Shifts

Widespread adoption and understanding of REST, truly without this we'll never get past generation three.

Data transformation, for a very long time data transformation will be an everyday service we all require, for legacy systems, legacy data, proprietry systems, data model conversion and so on - the key isn't to be able to understand all kinds of data / serializations / models, but to transform it in to what you do understand, easily.

The enabling of the Semantic Web, Linked Data provides the means to do this, it is the enabler - each EAV triple we put out is also a triple that can be reasoned over, full machine understanding of all our data is an ultimate goal that will take many years. Generation three very much pushes the semantic web in to the realms of the real world, no longer for the strictly academic.

The great cleanse, all the data from generations one through three (and possibly beyond) will need cleansed - as primary universal vocabularies emerge the task will be to clean what's on the web and migrate it in to the new models, reasoning and cleaning it as we go.

Cross compilation and seperation of syntax from machine code - another movement that is gaining momentum, no longer are we tied to a specific syntax for a specific JVM - not far off is the day where we can program in our preffered language, and compile it to whatever target we need. This is all ready happening in many quarters (like haxe) and soon will be the norm.

Seperation of human readable data syntax and data storage/transfer serialization. This has long been understood, however with EXI comes a light serialization of XML for the machine & transfer; logically from there we can expect the same in the future for RDF(2) and most data moving forwards.

Much of the code needed for the Application Tier will disappear over time, typically most application code involves taking some data and turning it in to some new data, processing it - when you can do this on the fly via data transformations, rules, reasoning, inference, querying and similar, you remove the need for most f the code in between. It is entirely possible that most server side web applications could disappear.

End

It is testament to the strength and design of the web that it can support every incarnation of the web thus far and in the future all at the same time - All credit to Tim Berners-Lee, Roy T Fielding and the many, many others who contributed, evolved and continue to forge this thing we call the web.

Disclaimer
Obviously, this is all just my humble opinion, none of it is fact - but from stepping back a bit this is what I conclude, for now.

I've skipped loads of things, most of the techs I work on and think about daily, FOAF+SSL, web access control, read write web of structured linked data, loads more - the above is just a summary of where I think we're headed, primarily for my own reference :)

Universal doesn't necessarily mean universal - but it's a good enough word to convey what I'm thinking. Any other use of terminology you don't quite agree with, likewise, terminology is not the point of the post ;)

Best & happy to hear your thoughts and additions,

Nathan


linked data extractor prototype details

I recently released a prototype linked data semantic extraction demo which combines OpenCalais, Zemanta and Openlink Virtuoso to effectively categorize and work out what a given peice of text / document is about.

OpenCalais and Zemanta usage details and service comparison.

The demo leverages OpenCalais in order to pick up references to things, which are returned in most cases as string literals; OpenCalais can also be configured to return back socialtags which give a broad stroke idea of what the document is about, again with string literal "tags". With regards the references (semantic metadata, Entities, Facts, Events etc.) which OpenCalais returns, whilst it is generally string literals, it also returns back vital Type and Relevance information, so in the case of "London" it will also assert that London is a City. Even in the case where it doesn't previously know what a thing is, it can work out that say "Frank Neverbeenheardofbefore" is a Person.

Zemanta is also leveraged, the primary difference between Zemanta and OpenCalais (and thus the need for both services) is that Zemanta focuses more on accurate tagging of text. Primarily though, Zemanta tags (again string literals) are meaningful tags which are commonly known and are referenced to either existing Linked Data identifiers such as http://dbpedia.org/resource/London and further information about the tag (or thing), in the case of the aforementioned London, then it will often also provide links to the wikipedia page for London, the official homepage to the city of London and a link to show the position of London on google maps.

I should point out that ever increasingly OpenCalais also returns back Linked Data too, for instance in the case of London they have given it an HTTP URI which can be dereferenced to retrieve more information about London. At a very crude estimation I would suggest that (depending on the subject matter) OpenCalais returns Linked Data URIs for about 15% of all references it finds to well known "things".

Weighing up the two services I couldn't say that one is better than the other, both have advantages and disadvantages, the only way to get a decent overall picture is to use both. for the benefits of feedback to both of these great services though, here is a general comparison:

note: none of these figures are from exact tests, they are from extensive developer usage of both services as I've used them both since they were made public.

Zemanta is generally 2x as fast for average texts (the size of this post for instance) and as much as 5x as fast for longer texts. Average for Zemanta being 0.7 to 2 seconds. Average for OpenCalais being 1.5 to 10 seconds. It may also be worth noting that the availability of Zemanta is somewhat higher than that of OpenCalais, perhaps 1 in 250 calls to OpenCalais will fail.

OpenCalais does a lot more heavy work than Zemanta though, and *really* semantically analyzes the text to figure out a wealth of information. In this respect the tables are completely turned and Zemanta consitently deals with providing a few high quality known tags; where as OpenCalais often provides at least 10x as much information about a given text, including relevance and type as mentioned before. OpenCalais also extracts Facts / Events, and further it can figure out that "Jim" is also "Jim Bob", and that Jim said X about Y on date D.

Generally you can trust the data from Zemanta 99% as it deals with "known" things, however due to this in some cases very new topics (such as IPad for the first few days after its announcement) remain unknown. Due to the nature of OpenCalais and it's dealing with the unknown you need to take more time to verify what it has returned, however when OpenCalais assigns a LinkedData identifier to something or provides more information you can 99.99% trust that it is entirely accurate.

It's worth noting that both of these services do different things though, and both do it extremely well, Zemanta "tags" and OpenCalais "semantically extracts information", in some respects I was hesitant about comparing the two, as in the context of what I'm doing both are needed and both are equal, however in different contexts both do different jobs and there is a need for people to select one over the other.

Out of all the competition though, I would highly recommend both Zemanta and OpenCalais over their respective competitors, and do hope that neither of these great services ever decide to target each others markets. (e.g. they compliment each other well and both do so well because they stick to what they are good at).

extractor.data.fm details

This demo deals primarily with figuring out what a document is about; in that it aims to provide back a list of:

  • Categories
    A list of 1-5 dbpedia (and therefore wikipedia) categories which the provided document would be categorized under if it were a wikipedia article and had been categorized by a huma who was knowledgeable in the subject domain(s) of the text.
  • General Topics
    A short list of the general and broad-strokes Subjects covered by the document, these can are distinct from the primary specific subjects covered and the categories, and in many ways can be seen as the most common intersections between the primary specific subjects discussed.
  • Primary Subjects
    These are the specific subjects covered in the document, not just the things mentioned, but the things actively discussed within the document, the primary subject matter as it were.
  • Related or Mentioned Subjects
    Whilst I've termed them "related" as in dcterms:related, these are simply things which have been detected in the document or text and which are not primary subjects; in many ways "mentions" may be a more appropriate term.

Out of the above list, the two services do the heavy lifting to give the demo it's Primary Subjects and Related Subjects; in short OpenCalais' SocialTags and Zemanta's Tags give us back our Primary Subjects. Whilst OpenCalais by way of the semantic extraction provide us with the Related Subjects, namely all those extracted semantics which have the Type of a real thing (not an IndustryTerm or Event) and which are not all ready a Primary Subject; additionally those extracted semantics which are not tags but have a relevance higher than a certain score are boosted up to be Primary Subjects too.

A primary and initial function of the demo is to associate the tags returned by both services together, and figure out when each is talking about the same thing; this is covered first by dealing with the linked data they return; where both services are talking about the same thing you simply know this unambiguously due to the nature of http URIs and them both being the "sameAs" each other. After this two chunks of unhandled data remain, Zemanta tags which have not determined to be the sameas OpenCalais ones; and OpenCalais semantics which we have a string literal name for and a type.

In step Openlink Virtuoso 6.1 (open source edition!) with most of dbpedia 3.4 loaded in to do the heavy lifting from here on; Virtuoso is a really powerful bit of kit and has replaced mysql/sql server/postgres, rdf store and web dav server in my typical server stack. The public lod and dbpedia endpoints really do no justice as to just how powerful and fast Virtuoso is, queries which take a few seconds on the public endpoint return in hundredths of a second on my local (low spec) server, and the comparative performance to the aforementioned RDBMS solutions is not to be sniffed at.

To handle the typed string literals from OpenCalais, I built a custom dbpedia lookup service (using sparql over the aforementioned Virtuoso + dbpedia setup) which tries to unambiguously determine the identifier for a string literal, if it is known; the results are pretty good and I'd safely say that it gets it right in 98% of cases. This essentially turns the remaining unknown string literals in to known Linked Data URIs, and as a side benefit gives the correct full Name for the thing identified along with the correct casing and obviously much more linked data.

Remaining now the demo has a few OpenCalais semantics which are still unknown, but we know the Type and have a name for the thing; and as URIs are given to things that can be Named, I simply mint my own uri's for these and specify the OpenCalais identifier as a "seeAlso" (to be future compatible with a time where they do associate there own hash uris through to linked data).

At this point the demo has all of the Primary Subjects and Related Subjects determined and where possible linked through to additional LinkedData and human readable web documents about the subjects.

Categorization

This is where the script comes in to it's own and really leverages virtuoso, up till this point it's all been about cleaning, validating, looking up, associating and suchlike.

Given that we now have linked data HTTP URIs for all the subjects we are dealing with, and in all Primary Subject cases we also have dbpedia.org URIs the demo can start to use some of Virtuoso's more powerful features. First point of call is to get the Category intersection of all primary subjects (including the inferred categories!) via a slightly complex transitive sparql query over the dbpedia dataset. From here the demo calculates a set of primary categories which the text is related to, then it finds the general category intersection (again including inferred categories) between the primary categories, and the primary subjects. with the results returned is a wealth of numerical information which the demo dually considers and can then infer which are the General Subjects and the Categories for the text.

At some point I'll cover this part of the script in more details and give some virtuoso specific transitive SPARQL queries for you to use in your own such creations, but for now the above will have to do.

Conclusion

This extractor demo is something I've been working on and trying to achieve for about 5 years, and whilst it is still early days it's the first time the technologies have been available to both make it possible, and to utilize the results correctly to achieve what I'm aiming for overall.

The overall goal is to create a system which allows users to simply drop in content, and the system "files" it in the correct categories, lists it under the correct subjects and interlinks it with other resource via typed links such as "related resources" and looser resource lists of "also mentioned here", further benefits of such a system are that you can accurately figure out what readers are interested in and promote new content to them, you can give users the option of content streams where they can watch specific subjects or combination of subjects to be notified of their "ideal" reading. On the flip side you can also identify users and contributers interests and expertise, and correlate these together (with geo-location) to suggest others users who they may wish to collaborate with, other organisations doing the same work in the same fields and many such uses. In reality I have much of this implemented in a site I've been working on for the last year, which is just being rolled out again, and the system works extremely well with huge benefits to all involved, the site you see deals with climate adaptation and both provides a service to the general adaptation community where they can share and find knowledge, and more importantly serves organisations working on critical issues by letting them see which people / organisations / projects are doing what, where and allows them to both co-ordinate efforts and perhaps more importantly not duplicate efforts and waste resources where it counts most. This has a positive impact on the worlds poorest nations and those suffering people who these organisations are trying to work with and help.

Back to the demo, and with the context described, the extractor.data.fm demo is a quick UI around an API which is in many ways the backbone of the aforementioned system. The API is used in a semi-automated way, where the data returned by it is verified in a UI by the content author / admins who remove any unambiguous data and then hit save, from there everything is automated again and the system functions as above.

I'm unsure whether this kind of system will ever be able to be fully automated (or whether its wise to allow this) as certain scenarios just can't be covered yet, a real life example of this is an initiative called "TEA", ambiguity at this level, and with entities which are unknown to systems or even the web of data, will always be an issue at some point, as things progress it may be they are only ambiguous once, on their first discovery, but that is still once; hence why this may always have to be a semi-automated process.


A lighter way to configure Apache for FOAF+SSL

Just a snippet post to say that I've found a lighter (and imho preferable) way to configure Apache to accept client side SSL certificates (with regards to FOAF+SSL).

The Standard Way
This way essentially exports all SSL data, certs, client and server side if you read the notes has performance penalty.

SSLVerifyClient optional_no_ca
SSLVerifyDepth 1
SSLOptions +StdEnvVars
SSLOptions +ExportCertData

The Lighter Way
This way simply passes in the SSL_CLIENT_CERT in to the env REMOTE_USER and skips the rest which you don't use (for FOAF+SSL).

SSLVerifyClient optional_no_ca
SSLVerifyDepth 1
SSLUserName SSL_CLIENT_CERT

Tested and works very nicely (again, imho).

note: Enabling SSLOptions +FakeBasicAuth will overwrite this with the Subject from the client side certificate.



  • webr3 avatar