IdeaDump - 24 Hours in

Yesterday I had an idea that I wanted to quickly write out and share in some kind of trackable way, where I could go back to it and edit + refine it later on. However, the problem was that a blog post or an etherpad didn't really cut it. So, I thought it's about time somebody made something to handle this, and without much thought quickly registered the domain ideadump.org, sent out a tweet to say just that and ask if anybody wanted to help, then started a quick etherpad to write down the idea, and then sent out another tweet asking for contributions of thoughts.

Here we are, under 24 hours later, and I'm very pleased to say that there's been a fantastic response and things are moving quickly, the site is up and hosted at ideadump.org, there's a github project, a twitter account (@idea_dump), a sketchy start at the UI - but, that's nothing compared to the invaluable feedback already given by the web community, just take a look at the openetherpad document! A huge, well thought out contribution from some great minds, people have spent practically all day contributing in one way of another, notable contributions from @izuzak @bendiken @manusporny @csarven @mhausenblas @jeffsayre @pimmhogeling @melvincarvalho @paulgeraghty - a huge stream of invaluable input that itself shows very promising signs for IdeaDump. Really, thanks to everyone who's contributed so far, it's been a great experience, and we're not even 24 hours in yet!

To make things even better, every part of the project is being created and released under cc-zero and unlicensed, so it's all in the public domain already and will continue on that way.

If you'd like to contribute simply follow any of the links here and start typing in the etherpad or the chat there, message me or anybody else who's contributed, send a tweet to @idea_dump or email hello at ideadump.org or myself - every contribution from a thought to an idea to a line of code is welcomed and encouraged - and if you don't like the domain or the way the projects going, or want to try out a different approach, just fork it, fork the idea, the code, the project, whatever you like, and of course we'll all be happy to help & converge things - let's expose the idea graph and help innovation in our sector along by getting some thought streams to explore.

Personally, I'm thrilled and would like to thank everybody so far, I'm thrilled because as you may have read earlier, I've changed my working model and taken a new approach to development and working in our sector - honestly, it was the best move I ever made, the weight has been lifted from my shoulders, I'm several times happier, everything is working out great from paid work to public domain work to home life to everything - essentially, that one change in the way I do things has made my quality of life exponentially better within just a couple of days, it's only the third day in and everything has sorted itself out, this project and post is just an minor indication of the impact it's had. Whoever you are I urge you to read my previous post and think about how you're doing business, go on, make the jump, it's nice over here.


a new open approach to business for 2011

A follow up from my previous post: "Where's the economic infrastructure for open source and public domain developers?".

From this point onwards, I'm an open research and development consultant, working under a new business model for our sector, a one which you will benefit from, regardless of who you are.

I spend all of my time in research and development, converging technologies and investigating new ones, a mix of reading, looking, trying, inventing, hacking, communicating, exploring, testing and refining, the breadth and scope of what I do ranges from overall web architecture down to optimization of a single line of code, and all in between - of course I have my specialist areas and fortes like many others, but generally it's the big picture which I focus on and specialize in.

However, I'm just one part of something much bigger, what makes my new job so interesting is the web community, the projects, the papers, the ideas, the specifications and the passions of others, even the tweets and the mailing list posts, my knowledge comes from you.

So, here's how the business model works..

Some of you need some of the information I know, some of the understanding, others of you are starting ventures and simply don't have the time to invest finding out about all the great projects and techs on the web, let alone how they fit together, some of you are working on those projects and techs out of passion and really need paid for your hard work and effort. Some of you need me to prototype out your ideas by converging technologies and getting a proof of concept going, some of you need the bug reports and patches, or even just the recognition. Some of you need me to talk to your development teams and give them feedback + point them in the right direction, technically or just at an overview level, to make sure your projects become a reality.

So, if I work for with you, and we use open source projects, we'll pay the developers of those projects in some way, we'll talk to them, help them improve their projects, pay for their time, pass on donations and sometimes hire them to tailor what they've done to suit your projects needs or to build the next version they've been planning, the one which has the features you need. And if you don't pay them or donate or give them recognition/thanks, whether that be money or something from their wishlist, something to make their life easier like a new monitor, or some form of promotion, then I will, and I won't work with you again.

Everything I do is open source, not just open but public domain, cc-zero'd or unlicensed, that includes my work on your projects, and yes others will get that work for free, and use it, and fork it, and that's a good thing, because it'll create a marketplace for you, ready made competitors, although of course you'll be ahead of the curve by the time this happens. Of course I won't be giving away your business secrets and game plans, or even how you're tying different things together, you keep your competitive edge and ideas, that's your intellectual property, but anything I work on with you is mine, and I give that to the web community.

I do not deliver finished products, nor do I maintain them, unless the task at hand is to refactor or optimize something. I will however work with your teams to ensure they understand everything inside out, and are happy to run with the project and turn it in to a finished product for you.

If I'm not the right man for the job, or we realize you'd be better with different technologies, then we'll find the right person, or people for you, and transition what we've worked out together over to people who can help you properly, and who specialize in what you need.

Can you hire me to work for you fulltime and exclusively? no to both I'm afraid, I need a stream of interesting projects and ideas coming through me to have any value to you, others and myself - and that's the way it's staying, you can of course retain a chunk of my time on a regular basis if you like though.

The work I'll do will continue whether I'm being paid by others or not, if you think you'll need anything from my generic to do list, or anything related, and you're a part of a corporate entity, then by all means sponsor me, or somebody else, to do them, if you're another person or like me and want to converge ideas thoughts and code, then let's do it. General rule of thumb when working with me from now on is, if you're getting paid to work on the thing we're planning or doing, then so am I, and if you're not, then neither am I.

As for renumeration on paid projects, of course it'll usually be a monetary value as with any consultant or contractor, but sometimes it'll just be a couple of days of talking, or laying out a project plan, turning out a technical spec or such like, when it's a smaller job then why not offer products or services or ask me if there's anything I'm needing or would like for me or my family, if there's a charity or an open source developer I'd like a gift to go to? I'll be doing the same from here on.

So, that's what I do from here on, I'm an open research and development consultant, an open developer, and a contributor to the web communities - I sincerely hope that many, if not most of you reading this join me in this approach, help this open approach to business in our sector along, and look forward to working with you, in whatever capactity that me be.

Thanks for inspiration

I'd like to thank a few people who've led and encouraged me to take this step, Melvin Carvalho who whilst being an open source developer himself, also takes the time to help others, donating everything from time to funds to help fellow developers, he's been a real inspiration to me, likewise Sir Tim Berners-Lee who also leads by example, he not only invented the web but ensured we can use it royalty free, still finding the time to develop open source projects to this very day. I'd also like to thank Richard Blakely and Influxis for both encouraging me to take this step and for being the first to back me in working this way.

Many others have led by example, open sourcing all their work for us all to share, some shining examples are Arto Bendiken, Manu Sporny and the team at Digital Bazaar, the Ushahidi team, the World Wide Web Consortium and those who tirelessly dedicate time to standardize and improve the web - and finally, a thanks to all those who open source, share knowledge and help advance the human race.

All the best for 2011.


Where's the economic infrastructure for open source and public domain developers?

2011 is just around the corner, I've got a huge list of things to do which I could reasonably and easily spend all year doing, I'm quite sure (hopeful?) that they will all have positive economic impacts for many people, many of whom I'll never know, in the same way a library like jquery does, or any other open source / public domain script, any spec, hack or innovation in our sector.

So here's the problem, who pays? the reality is we're all humans with needs, many of us have full families to support and raise, the work we could and want to be doing is far more valuable to the web at large than simply making another "website" for a paying client, yet the common variety website will pay us well personally, and the valuable open source / public domain work will pay us zero.

Then there's licensing and copyright, essentially I (like many others I'm sure) simply want to open source and cc-zero every single bit of re-usable or reference-able work I do from here forth. In previous years I've managed to strike a bit of a balance where I picked projects based on things I felt were needed generally, or which I wanted to learn, I could release the generic code created and give clients the application specific code, however, increasingly I find that the work I do provides the framework for projects and applications, but not the application itself - just take a quick look at my generic to-do list and you'll see what I mean.

So, what do we do, what do I do? How do I get to this time next year with all these things done, released as a source of reference, as usable libs, and as prototypes, whilst still being a "working man" rather than charity worker for some invisible charity?

This is a big problem our sector needs to address quickly, kickstarter and the like doesn't address it, neither does public domain funding (I'm not a "business" I'm a "me" with no plans to start a fully blown company, tis a waste of all our time if I head down that path). Somebody mentioned a return of the patronage model the other day, maybe linked with micro payments, but until that's common place it feels a bit like scrounging / free loading (even though it's not really).

At the end of the day, I could easily spend the next year doing 100+ hour weeks of v hard work (just like last year), but there's no client. Something has to give, or change here - I along with many others don't want to slow down all of this or even stop all together, but one has to provide for their family.

What do we do, what's the solution here for the many people out there like me?


Generic to do plan

Personal high level to-do list for the next x amount of time ahead...

To do:
Update rdfa-api to align with the RDF API, add full Graph Literal support, add RDFa 1.1 Core parser + profile support, finish generic HTTP based CRUD clients, merge with JS3.

Upgrade to support full N3, work on rule, reasoning and inference methods with OWL and rdfs support + sl/ql/fol based approaches, basic and improving understanding of ontologies, including restriction and owl based data validation.

Spec and upgrade to RDF Object View, spec and implement bi-direction oo-friendly tree views of RDF.

N3 Paths, RDF Selectors (and maybe SPARQL implementation?).

Create hook-in predicate, class and rule based agent framework for data streams and graphs, on top of aforementioned tooling - agents which listen for specific types of data / sets of statements and then do what's needed - to handle simple reflex, model-based reflex, goal-based and utility based intelligent agents.

Combine intelligent agent framework with data wiki support to create intelligent cloud storage agents, personal data streams and helper agents. (agent per uri, each agent has a task, collection of semi-dumb agents in to a data space makes a semi-intelligent agent).

Work on double-agent based auth/ident protocol - four-party, decentralized, temporally valid, webid-like protocol, with built in foothold for trust - details soon.

Leverage HTTP POST for storage / triple accepting agents in a "document which accepts annotations" style. Implement + compare and contrast SPARQL update vs Diff/Path approach.

Web Socket based triple & rdf object streams for personal data streams, filtered data streams, data synchronization and updates.

Generic linked data client side agents for specific tasks - basic social clients (messaging, friend adding, personal data management), and generic apps (like good-relations based client side basket which works on any site), shift of web application state to client side w/ stateless protocols and semi-intelligent agents at both sides.

side topics:
- micropayments via payswarm, open transactions, bitcoin.
- convergence of RDF libs and tooling for interoperability based on standardizing apis.
- p2p web / web architecture & movements on persistent domains, alternative approaches to resolvability/dns, bidirectional/back links.
- More amaya-like browsers/user agents, desktop based http client+cache+server in one efforts.
- RDF serializations (Turtle+Graph Literals ala AMORD in RDF, JSON based serializations / json-ld).

primary long term focus being the convergence of:
- read write acl enabled web of data
- "intelligent" / semantic agents
- internet of things
- augmented reality and innovations/improvements in device + human interface sectors.


RDF API, JSON Serialization and Standardization

Since there's been a lot of discussion about JSON serializations of RDF, and the need for an RDF API, I thought I'd offer my own personal thoughts on what we need from a JSON serialization and an RDF API.

A JSON compatible serialization of RDF

JSON is a lightweight text-based open standard designed for human-readable data interchange.

That's not what we need.

We need a lightweight text-based JSON-compatible open standard designed for machine optimized RDF data interchange.

We need it to be JSON compatible because JSON tooling is widely implemented, JSON is well supported, embraced by most web development communities, and of course it's fast and simple.

Stress: It won't be JSON, it'll be JSON-compatible, and to avoid any confusion or indicating that it may be human friendly, we probably would be wise not to put the term "json" in it's name, although we will need to put it in the media type identifier "+json".

We need to standardize because there are numerous competing and differing JSON serializations at the minute, so when somebody asks for a JSON serialization on the web, they get back different results in different places - that's unexpected behaviour, and it's not interoperability - so we need to standardize a JSON-compatible serialization of RDF, with an IANA registered mediatype.

The serialization needs to be machine optimized and usable in multiple different scenarios, else there's no point standardizing it, because other JSON compatible serializations will still be needed and therefore created. Thus we need to define the various use-cases and create a JSON-compatible serialization which caters for them.

An RDF API

There's been a lot of call for cool RDF tooling, specifically in javascript, mentions of a jquery extension and a jquery-like library - I agree that would be nice, but also say why just one? I like and want choice, freedom and to see innovation in the marketplace, I don't just use jquery, I use more libraries and modules than I could name - however..

A library is not an API, and that's not what we need to standardize.

We need to enable interoperability between libraries, and where multiple libraries may or do have overlapping and conflicting functionality, we need to align and standardize it, but only when it produces unexpected behaviour or limits interoperability.

Primarily, this means standardizing interfaces for the RDF Concepts (literals, references, triple and graph) - as soon as we do that, all libraries which implement these interfaces will immediately become interoperable. Web Developers will be able to throw data between libraries, cherry picking the features they want or the coding styles they prefer, library implementers will be able to focus on areas they specialize in, others will be able to stun us with innovative new approaches and tools we'd never have thought of ourselves, and we'll have an open and ever growing truly cool suite of interoperable classes, libraries and tools.

Next up we can standardize interfaces for things like RDF Parsers, because as soon as we do that, all libraries on each specific platform can share parsers, and these can be developed, maintained and improved independent of any specific library - it will also allow the same libraries to be used with specialist low memory parsers optimized for mobile devices, thus broadening their potential user base and giving the end users + developers the best experience possible.

Another set of interfaces which need standardized, are those which are to be implemented by multiple different vendors on the same platform, i.e. the web browsers, i.e. anything near the DOM or HTML, i.e. the RDFa API - which is already being worked on.

Finally, an interface which will make linked data and RDF easier to work with by developers (carrying on from my previous post) is a "Data Object" interface, that is a simple object which holds the description of a single subject, where the properties are keys and the values are native values. I was wrong in my previous post, serving up simple JSON objects will help, but isn't perfect, because we don't know and can't determine whether people will want to use URIs, CURIEs or terms to access properties, thus an interface will be needed to handle this.

This isn't an exhaustive list, and each paragraph above can be considered to handle a different layer in the stack, but standardizing anything substantially more than that will stifle innovation and be counter productive, people need and want different libraries with different coding styles, features and different ways to access and query data. Different tools for different jobs, and for different people doing the same job.

If you disagree, please do share and let me know!

Clarification

Earlier this week I was quoted as follows:

Nathan’s recent post on Opening Linked Data, which is worth reading in its entirety, captures the essence of the issue:

You can’t shoe horn RDF in to JSON, no matter how hard you try - well, you can, but you loose all the benefits of JSON in the first place, because the data is RDF, triples and not objects, rdf nodes and not simple values

In other words, using JSON as the basis for an RDF syntax doesn’t actually win you anything in terms of the ease of processing of that RDF. In fact, I’ll go further and say it has exactly the same bad qualities as RDF/XML.

However I have to point out that I probably didn't make the context clear, I was referring specifically to making a human friendly/optimized JSON serialization of RDF, you can however, very easily, create a machine optimized JSON-compatible serialization of RDF, without the drawbacks of XML (you just don't pin it as being human friendly JSON as outlined above) because unlike XML which requires a full XML stack, and RDF/XML which can be serialized in any one of a billion ways, we have a chance here to make a clean lightweight unambiguous serialization of RDF, one which is based on a lightweight data interchange format and not on a heavy extensible document interchange format.

Hope that clarifies :)



  • webr3 avatar