An update; sort an explanation of why I haven't posted anything recently.

I really want more control over my blog data.

I think Linked Data is great and the only way to make it more useful is to use it more.

I'm sick of Blogger's JavaScript heavy wysiwyg editor that doesn't work properly on low-bandwidth connections, or on my Kindle in emergencies.

I want to author blog posts in markdown, add whatever metadata (like tags) I need, and publish them via whatever means I have to hand. From a browser, from a text editor and command line, from an email, from a tweet?

So I'm working on Slog'd: Semantic Blog from Markdown.

I already wrote a wee Python script to convert my existing Blogger posts to linked data: Blogger2LD. I'm making a web interface for this so you can do yours... and I'll do it for Twitter, too. Then who knows what else will follow.

I'm still twiddling with Slog'd itself - which I'm writing in PHP and storing triples in a MySQL database using the ARC2 library, because I want anyone with any bog-standard shared hosting and no understanding of linked data to be able to use this. You can watch progress on Bitbucket.

I've been blogging every detail as I go along. These are in the posts directory in the Slog'd repo, rather than on here.

The post I wrote about EdinburghApps, the City of Edinburgh Council's awesome hackathon, is on the OKFN Scotland blog!

Writeup imminent...

Young Rewired State is a week-long hack event for under 19s.  There are centres all over the UK, and the week finishes with a giant sleepover in the Custard Factory in Birmingham, presentations and prizes.

I was helping out with running the Edinburgh centre this year, between the 5th and 11th of August.  We had 15 young people taking part, and a few parents popping in and out as well.  Not to mention several fantastic mentors.

Every day we gathered in one of the University of Edinburgh Informatics computer labs.  On the first day we did some brainstorming, introduced the young people to Open Data, and they sorted themselves into teams.

We had a diverse range of projects by the end of the week.

The Weatherproof app was written in Scala with a Web frontend, and as well as telling you the weather forecast, gives you practical advice on what to wear and what to take with you.

Stuff Index was a Python Web app that lets people photograph and upload stuff they've left out on the street that they want to get rid of, so anyone browsing the site can opt to take it away if they fancy it.  Helping to keep stuff out of landfill, and without the dreaded social interactions that come with Freegle.

Tag is a game by a one-man team, with a Python game server and a JavaScript front end that lets you chase your friends around the real world, and automatically tags them when you're in range.

PokeGame is a real-world Pokemon simulator that lets you roam IRL and capture virtual Pokemon.

Great stuff!

On Friday we crammed into a coach along with the participants from Aberdeen, Dundee and Glasgow, and set off on a seven hour road trip to Birmingham for the finale.

The Edinburgh teams didn't win anything, but the presentations were fantastic and everyone had an amazing time.  The young people made new friends, learnt tons of new stuff, and hopefully remain enthused about coding.

Next year we're going to do more to walk through the creation process of some example apps to get them started off, and maybe do a better job of introducing Open Data and the possibilities it holds.

We're also thinking about starting a regular under 19s code club in Edinburgh - weekly or bi-weekly - so stay tuned for more info about that.  (And if you want to help or participate, get in touch!)

Thursday 16th of May was the 6th Open Knowledge Foundation meetup in Edinburgh.  We had a great room in Techcube, more speakers than usual and loads of attendees.  Here's an account.

Bill Roberts

Bill, founder of Swirrl ("the linked data company") talked about tools and user interfaces they are developing to make handling data easier for communities; particularly for the less technical.  They've encountered a spectrum of users with different levels of technical abilities and needs, so they have to account for this in the tools they build.  Technical complexity for accessing data ranges from SPARQL endpoints, JSON APIs, downloadable spreadsheets to visualisations, maps and charts.

They're focussing on providing the data in accessible ways rather than building visualisations though.  They'd struggle to meet or even understand everyone's needs; instead, it's important to concentrate in empowering the communities to use the data themselves.

Kim Taylor

An undergraduate Informatics student and participant of the Smart Data Hack, Kim showed us placED, the project her team had worked on, and are continuing to develop.

This is a place finder for people who are looking to maintain or improve their personal wellbeing.  They used datasets from the City of Edinburgh Council (and presumably ALISS?) to create an Android app. They stored their data in a Google AppEngine datastore, but I'm not sure if it has a web frontend as well.

Some of the problems they encountered include copyright issues with ordnance survey place data, and royal mail postcode data, which made up part of the Council's data but wasn't available for anyone to use due to licensing restrictions.  They worked around this by recomputing location data from the parts of addresses the did have access to with Google's geocoding API.

When they open their database up for user input, which they inevitably will if they want their app to stay current and useful, they'll have to think about how to maintain the content.

Gavin Crosby

Gavin works for the Council, with a title I've forgotten, but it's to do with youth work.  Youth work has a very specific definition to do with people aged between 11 and 25, meeting in organised groups with a volunteering adult present.  There's loads of this going on in Edinburgh, organised by Scouts/Guides, schools, churches or maybe even self-organised.  There's no central database about what is going on where, which is one of the Council's biggest issues in this area.  Word of mouth is usually how this kind of information is spread amongst young people, and Gavin suggested that a lot of youths may be unwilling to attend something they'd heard about without a direct invitation from someone they know.

In an attempt to reign some of this information in, they've created the Youth Work Map.

It's not an ideal system, as they have to update it manually when a youth group or activity organiser decide to inform the Council that they exist.  Not everybody opts in, so there is data missing.  Manual updating also means the map is not 'live'; things might go out of date and not be removed straight away.

Gavin said it is the constraints of the Council's web system that has caused a lot of the problems, and points out that they haven't considered accessibility issues (for example, access for people with vision problems), and it's not interactive.  He'd love to see the ability for kids to chat to each other through the map, or leave reviews for particular events.  There are issues with child protection here, of course.

He would also like to see better tagging and organisation of the content on the map, links to other data repositories (there are parallel similar projects), and the ability to connect events to areas or routes rather than single points.

Gavin pointed out that a lot of the audience for this map is likely to be adults looking for youth projects, rather than young people themselves.

Leah Lockhart

Leah made a quick announcement about the new Local Government Open Data Working Group.  They're organising open data surgeries (similar to her social media surgeries that you've definitely heard of by now if you're floating around the OD scene in Edinburgh).  They're also hoping to fill in the OKFN Open Data Census for Scotland, and meet regularly in the pub.

Tweet Leah if you're interested!

Fiona McNeill

Fiona works in Informatics at the University of Edinburgh, and she told us about Open Data in climate change science, or the lack thereof.  A team she has put together have got some funding to carry out a small investigation about Open Data use in climate change science, and to try to build a network around this.  They'll be looking at trends and patterns of the past decade to see if research has been any more successful when existing datasets were used, or if papers are more well-cited when they make their data open at the end of it (for example).

She thinks the lack of Open Data in this area could be due to the expensive nature of making data good enough quality to share, and of course the fact that when people have worked hard to gather data they feel that they own it; why should they share?

They're hoping that their report might go some way to persuading funding bodies to have sharing of data as a criteria for applications.

Contact Fiona if you're interested in this kind of thing.

John Kellas

John said, brilliantly, that talking about information visualisation usually means graphs.  But normal people "don't think in graphs".

He works in community education, and a couple of years ago he started working in "volumetric and comparative" visualisations, which can be much more useful and empowering to people.  He showed us a visualisation of one trillion dollars (which I can't find a link to, so let me know if anyone has one).

He's not had much support with creating tools and visualisations, because he's not interested in making money from it, so it's hard to attract funding.  What he's doing looks really useful though, so hopefully we'll see more!

Ben Jeffery

Ben is another undergraduate Informatics student who took part in the Smart Data Hack and whose team is still working on the project they started at the hack.  They're re-imagining the University's student information portal by pulling in lots of different data sources, and presenting the information more sensibly.  They've been doing a fantastic job, but of course are all busy with exams and general learning, so the haven't been able to spend as much time on this as they'd like.

They're also struggling to get raw data out of the University, and point to (my alma mater) the University of Lincoln's open data portal as an example of what could and should be done about this.  So they're turning their project into a pilot to demonstrate what they could do if they had the data they wanted.  They're also conscious of similar-but-different projects, like projects.ed.ac.uk, and don't want to duplicate effort.

Ben said they've found that a lot of the University of Edinburgh's data is held by middleware vendors, so it's particularly hard to access.  But this is information that is funded by students, so it should be available to them!  He said the "University should be a breeding ground for knowledge" so data shouldn't be silo'd up.

He also said that there are a lot of politics in the way with this sort of thing.  They, as any level-headed software developer, just want to build stuff.  They're still in various talks though, so this is a space to watch...

Susan Pettie and Marc Horne

These guys are from So Say Scotland and aim to change culture to make Scotland better.  Open Data is important for democratic movements, so they told us about some of their events.  They're building a network of activists and campaigners, and hold large scale assemblies themed around 'thinking together', which is a kind of en masse guided brainstorming.  They're trying to spark a movement, and are aiming for 25,000 people.  They're investigating ways to make their assemblies more efficient, as currently collating all of the ideas that are generated is a manual process.  This would be nigh on impossible when they reach their participation goal.

There will be a report about their progress on the 27th of May.

Devon Walshe

Devon was our Techcube host, and he told us about Sync Geeks, Geeks in Residence.  This is a program funded by Creative Scotland that puts the technologically minded into arts organisations.  Previous efforts by arts organisations to employ 'geeks' to solve a technical problem or produce a digital solution for something have been problematic due to the 'black box' approach.  The developers produce an outcome, get paid and leave, often aiming to do the minimum amount of work.  Geeks in Residence promotes developers and the organisations working together more closely, to allow for sustainable solutions.

Part of the project is to analyse the relationships of people who know about technology, and those who don't, with each other.

In my notes I've scribbled "convert fear into technology", and I can't remember what that originated from, but it sounds awesome.

Devon did some work with Stills photography centre.  Nobody knew what they needed, so after some collaboration they developed an interactive floor plan (because the Stills building is way confusing) and some kind of interactive timeline because Stills has an interesting history.

He also plugged the Culture Hack Scotland in Glasgow in July (12th-14th), which I'm terribly disappointed I won't be in the country for.

Next OKFN meetup

Will be on the 22nd of August, in Informatics.  Here's a link to the Meetup so you can RSVP.  I'll sure be there, if I'm not somewhere else.. (depends if/when/where my Mum books an obligatory family holiday).

The 23rd of February was International Open Data Day.  Since this fell at the end of the ILWhack week, it seemed like a good opportunity to take advantage of the momentum and engage the stirring Edinburgh Open Data scene.

Many ODD were organised around the world; most of them were hacks.  In Edinburgh we went for a different approach, thanks largely to input from local community activists like Freda, Leah, Andy and Ally.  Some might call it 'social hacking'.  Our aim was to gather together people with little knowledge of Open Data, people with data that may or may not be open, and developers and technical types.

With Open Data stuff exploding worldwide, and developers going nuts creating cool apps and services that make use of data being released, it's important, you see, for {local, small-scale, voluntary, grassroots} groups and individuals to get involved early on.  Such parties are arguably likely to benefit the most from empowerment by data, and if they're not part of the discussion early on we might well see a lot of services developed that meet needs imagined by a not-quite-connected but well-meaning tecchie.

So on that note, we want to spread the word about Open Data to those who might normally be left behind.  When these groups know what the possibilities are (we can show them successful projects, locally (eg. ILWhack) and worldwide), what is available, and what could and should be available, we empower them to take action that will benefit them.  More specifically, people can find out that the English government has released data about such-and-such-of-interest, and politely demand that the Scottish Parliament or local councils do similarly.  They can interface early on with developers who are keen to start making, and make sure their real problems get solved (or at least prioritised over potential imaginary ones).  They can get involved with things like ILWhack, and have a better idea of what it's all about.

Between 10am and 2pm, we gathered around 35 people in the Informatics Forum and, fuelled by tea, coffee and biscuits, began the discussion.

We started with an hour of ten minute talks, about a variety of topics:


Sally Kerr told us about Open Data at the City of Edinburgh Council; the progress they've made so far and where they hope to go in the future; NESTA's local government Make It Local programme helped Edinburgh Council to move forward with Open Data.  She gave a nod to the ILWhack projects that made use of Council data in the week prior.

Alex Stobart of MyDex talked about big, open data, and the challenges this presents to citizens and politicians.

Iain Henderson explained the Standard Label; an easy to read specification for data holders to present to their users how the user data will be used.  Like nutritional advice, but for data.. Other ODD events were centred around hacking with it we spoke!

Bob Kerr talked about OpenStreetMap and GeoRSS.  I love the obsessive hyperlocal detail in some places, like where the animals live in Edinburgh Zoo.  On a serious note, OSM has really empowered local governments and NGOs in developing countries.

Andy Hyde discussed asset mapping for voluntary groups; how ALISS collate dispersed health and wellbeing information into a central, open repository, ripe for manual and programmatic access.

From Lizzie Brotherston we heard about the Post-16 Learner Journey Project; helping the Scottish Government understand the learning landscape.  They're holding a hack in April.

Next it was unconference time!

We had a short while of whole-room discussion, before identifying three key areas:

• Standardising visualisation (headed by Bob Kerr)
• Small scale voluntary organisations (headed by Leah Lockhart)
• Sustainability of data projects (headed by Ewan Klein)

Everybody picked a group and we broke apart for the next couple of hours.

The final part of the day was a return to the main room, and further room-wide discussion of the breakout debates.

The standardising visualisations conversation focussed around bringing people into conversations about data using visuals.  Someone pointed out that if news readers used Open Data visualisations, the general public would be a lot more interested in Open Data.  It's interesting to imagine a future where data visualisations are embedded into the world, into the landscape.  To be able to interact with data meaningfully, you've got to know what it is - to recognise it.  A standard - think periodic table - would help people to know exactly what you're talking about straight away.  This goes beyond graphs and charts, into a world of layered visualisations that allow layered public contributions of interpretations.

Those interested in small scale voluntary organisations discussed data holding and data access issues, including strategies for persuading big organisations to open data (eg. by showing success stories, and proving a certain return on investment).  It was agreed that interfacing with developers is important to get things done that organisations really need; but organisations might not know what they need.  It was discovered that there's a lot of crossover between groups represented by people who were in this discussion; common needs but gaps in talent.

Finally, with regards to sustainability of data projects it was agreed that strategies are needed for keeping things going beyond short hack events; how to sustain that burst of energy for a longer term usefulness?  How to keep track of everything that's going on, and link communities with events (see OpenTechCalendar!).  Some kind of coordination body might be useful, or working groups / task forces.

We wrapped up, collected everyone's details for sharing (to ensure sustainability of the outcomes of the day by making sure everyone can keep in touch!) and people began to drift away.

There was an enormous positive energy throughout the day.  Discussions were lively and passionate, and we had an excellent mix of people, exactly as hoped for.

NB. It looks like Joined Up Edinburgh will come under the umbrella of the Scotland branch of the Open Knowledge FoundatioN, so http://scot.okfn.org will be a good place to keep an eye on now.  And to keep in the loop, join the Joined Up Edinburgh mailing list.

Other people have blogged about this too.  Check out these by Leah Lockhart and Dave Meikle (more links welcome).

I spent a good deal of time during January and February helping to organise a couple of Open Data oriented events.  At least, that's the excuse I'm sticking to for not having done much of my PhD in that time.

The Smart Data Hack, also known as ILWhack (Innovative Learning Week* hack) came first, between the 18th and 22nd of February.

* Innovative Learning week at the University of Edinburgh is an annual week of off-timetable activities for students, designed to enhance their learning experience.  Arguably every week in higher education should be innovative and striving provide the best possible education...  And Innovative Learning implies the student should be making the special effort and I don't think many would be happy with the idea of paying so their tutors can have a week off, so maybe Innovative Teaching week would be more... better.  But that aside.

The hack was targeted at first and second year undergraduates in Informatics on the basis that third and fourth years would be busy with final projects.  This was by no means a restriction however, and we harboured hopes of enticing along design students and data buffs from other departments to mix up the skill set a bit as well.

I knocked up a website with two primary functions.

1. Students could pre-register, add some info about themselves and start to form teams online.
2. Anyone interested in getting involved who wasn't a student could figure out where they might fit in and get in touch.  This included people who could sponsor prizes, present real-world challenges to solve, offer data to be wrangled, or provide technical support to participants.

We anticipated about 50 students, and invited them to form teams of up to 5.

In parallel with gathering sponsorship, we came up with five prize categories of equal merit:

• Best for travel
• Best for health and wellbeing
• Best for communities
• Best visualisation or UI
• (First year prize for) Best data mashup

We hoped to encourage students to make whatever they wanted, using whatever technologies they wanted, with use of open (or specially provided) data being favourably looked upon. 

Skyscanner were the first main sponsor on board, pledging prizes for two categories and some massive datasets that aren't usually public and access to internal APIs, as well as engineers to mentor.

We partnered with ALISS to encourage use of their local health and wellbeing data API; ALISS also sponsored in part a prize category.

The City of Edinburgh Council were on board with some never-before-seen downloadable datasets (still online!), a bunch of pre-approved API keys and refreshingly open minds and supportive attitudes.

CompSoc heroically sponsored an entire prize category and promoted the event to its members.

Open Innovation sponsored a prize category too, and the School of Informatics contributed towards prizes and catering.

Greener Leith proposed a challenge and sponsored a special Mosque Kitchen lunch for everyone after the mid-point presentations on Wednesday.

We were able to hold some terrific practical workshops, thanks to:

• Tom Armitage and Stuart MacDonald, for handling geolocated resources.
• Philip Roberts, for data visualisation with d3.js.
• Oli Kingshott, for an introduction to version control, and HTML5 for beginners.

We also recruited mentors from UG4 and PhD students, as well as industry professionals, who were consistently present in the hacking space all week or available by Twitter, email and IRC.

We marketed the event in the couple of weeks prior (though we were organising up to the very last minute) through shout outs in lectures, posters around the Informatics department, emails to many university mailing lists and word of mouth.

As a result, we overshot our expected numbers, with well over 100 sign-ups by the start of the week.  This was good news and bad news at the time, as we had to scramble around to make sure we had enough sponsorship to feed everyone and whatnot.

By the end of the week, there were around 80 students still actively participating, across about 25 teams.  Pretty good!  Most of them were Informatics undergraduates as expected, but we had a handful of postgraduates and students from the ECA as well.

And the outcome?

Some amazing projects and really positive feedback from participants and supporters alike.

Naturally only a couple of days passed before somebody noticed that I hadn't sanitized input fields on the website for HTML and CSS input, so they made the projects page spin and play the Harlem Shake before I sorted that out, having been alerted at around midnight. /grumble.  Should have seen that coming, of course.

In the end we gave away £1500 in Amazon vouchers, five Nexus 7s and ten Kindle Fires.  Skyscanner even upped their sponsorship to three prizes because they were so spoilt for choice.

You can read about the projects, teams and see some demos here.

It was a really exciting and inspiring week for everyone involved.  Many of the students are taking their projects further (which is probably the most important outcome) and are in discussions with relevant parties to do so.

You can see photos and see other peoples' write-ups about the event here.

Will we do it again next year?  From the feedback gathered, the response has been a resounding yes!

Tell Me Scotland publish public notices for things like traffic, planning permissions... And they have a SPARQL endpoint!

I'm hacking around with the ultimate goal of creating an interface that allows people to generate a GeoRSS feed for a particular area.  Ally of GreenerLeith suggested this, so that they can use this to feed into their own apps.  A stage beyond that is to smoosh the whole lot into a Wordpress plugin, to make it accessible to anyone (who uses WP).

So far I've got the notices on an OpenStreetMap.  I haven't had a whole lot of time, but will make more soon.

I'm using the PHP library ARC2 to deal with the linked data.

NB. the TMS endpoint is, at the moment, flaky at best.  I think they're working on this.

11th to 24th February

I went to the Computer Mediated Social Sense-Making workshop; read about it.

I helped organise and run the Innovative Learning Week Smart Data Hack (18th - 22nd), and Edinburgh's version of International Open Data Day (23rd).  Both were successful. 

Which was hosted by the National Library of Scotland. (Information).

I reported on the 1st International Open Data Dialogue in Berlin that I'd been to in December, but then had to immediately leave, so I don't have any notes on the rest of the talks..

7th January - 13th January

I was subject of Paolo's procedural task labelling pilot experiment.  It was quite complicated.  I don't know if I did a very good job.

I went to TechMeetup and networked like crazy.

• I talked to Andy Hyde about ALISS getting involved with the ILW hack, and Edinburgh OD scene in general.
• And a few other people about supporting the ILW hack in various ways.
• I talked to Felix Gilfedder of Popcorn Horror about novel ways of engaging with creative digital media communities (PhD related!)
• I briefly met Russell, with whom I hope to discuss digital video metadata (PhD related!)
• One other thing which isn't related to Open Data or my PhD but is very exciting nonetheless.

I went to a Design Informatics talk about the Internet of Things.

17th December - 23rd December

Mostly things not strictly PhD related...

18th - Initial meeting about organising a hack for undergraduates during Innovative Learning Week in February
19th - Discussed how local small scale community groups can benefit from Open Data, and how they could get involved in the ILW hack, with Freda O'Bryne and Euan Jackson.
        - Also discussed these things with Ewan, and started on a spreadsheet of decent apps that use Open Data, and links to their data sources.

Read my complete notes from day one, and complete notes from day two.

The 1st International Open Data Dialogue in Berlin in December was broadly a discussion about real-world applications of Open Data.  Lots of practice, less theory.  Despite this (or perhaps because of this, now I think about it) it wasn't as technical as I expected.  Felix Sasaki [1] talked about some basic technicalities of Linked Data and the Semantic Web, kind of the first things you'd learn if you were studying it in a structured way, and I heard a lot of people afterwards complaining that that had been too technical.

Importantly, there was a real message of getting things done at this event, and plenty of evidence that a world built on Open Data is not an idealistic pipe dream, but a reality right now.  Challenges are being articulated, and solutions are being created, and problems are being overcome.

I stress this particularly because a couple of sceptics who weren't at the conference tweeted things along the lines of "Sounds like your conference is a bunch of idealist hippies preaching to the choir…"  A genuine concern, but what's really exciting is that this definitely wasn't the case.  It was instead a bunch of realist technologists with the expertise and influence to actively overcome barriers to improving the world.

Open Data is about social change and empowerment.  It is about accountability of organisations with massive influence over the lives of ordinary people.  It is not about an abandonment of personal privacy, or everybody knowing everything about everyone else.

It should go without saying (yet it still needs to be said) that it is not appropriate to blindly make all data available to everyone about every aspect of everybody's life.  But what if you had access to all of the data anyone had ever collected about your life?  Think about purchase history (shop loyalty cards, travel tickets), online activities (searches, browsing history, social networking).  All this stuff is being stored anyway, all over the place.  Often by organisations who fully intend to profit from it, presumably with your unwitting consent.  They went to the trouble of collecting it, but you went to the trouble of providing it.  It's your data too.  What could you do with it (or hire a software developer to do with it)?  Then imagine you had access to the same data from everyone in your town, aggregated and anonymised, and visualised in a nice way.  Maybe you could team up with your neighbours for cheaper bulk food purchases?  Maybe you'd realise that others had similar hobbies or problems nearby, and could form special interest or support groups?  Reduce costs by sharing transport to similar destinations (or just have some company on the journey)?

There's so much potential within data that's already held.

The UK government's Midata initiative is a massive step in the right direction [3] toward compelling commercial enterprises to hand over machine-readable datasets to consumers upon request.

In Slovakia and Kenya (and possibly others, but these were the ones that came up), there is a constitutional right to data held by the government.  Not without loopholes and other problems, of course [5, 2].

One of the obvious problems is convincing large organisations that hold lots of data (like commercial enterprise and governments) of the circumstances in which it would be in everybody's best interest to release (some of) it.  Reasons they don't include a lack of understanding of the benefits; disproportionate assessment of risks; aversion to change; a lack of technical expertise and infrastructure; "data hugging syndrome" [2]; licencing issues; outdated business models.

Nigel Shadbolt's experience says that large organisations who open data always see benefits.  It's always worth the effort.  When the data is there, suddenly developers start doing things with it; applications appear, many unexpected, and usually free.  He stressed that it's important to have a stockpile of success stories in case you need to convince someone in charge of the value of Open Data, and his favourite one was the publication of MRSA rates in hospitals (resulting in sharing of good practice, and an 85% reduction in MRSA over two years).  See a list at the end of this post for all of the success stories I came across over the course of the two days.

There were lots of discussions about the users or audiences of Open Data, and the various different roles people can have.  Most consumers of Open Data are developers, and 'ordinary people' see the data via an application.  Many won't know (or care) about the source of the data that powers the app, even if it about them.  Many will, and trust must be built for people see the value that such apps could bring to their day to day lives.  Ideally, releasing a dataset would be part of an ecosystem, rather than a one-time thing.  Data providers should value consumer feedback, and commit to good quality, up-to-date data.  Rufus Pollock wonders why every dataset doesn't have a public issue tracker, and notes that poor quality data creates wasted time, especially at hack events [4].

A successful Open Data world needs partnership between the public, media and organisations.  All of these parties need educating on appropriate combinations of the realistic potential of Open Data, and the technicalities of releasing and using it.  Michael Hörz [6] discussed the journalist perspective on Open Data; they're desperate for data about everything, and often manage to get hold of it.  But they find themselves begging for spreadsheets or CSV files, because what they get given are PDFs.  Eugh!  Yet they're not asking for Linked Data formats?  Which means, presumably, that after they've been through the trouble of extracting data from PDFs, they're putting it in a spreadsheet or something, and there's still a whole level of usefulness missing.  And I assume that's because they don't know otherwise, or perhaps don't have the resources to learn even if they're aware of the possibilities.  Similar sorts of reasons that they're being given PDFs by organisations in the first place.

So awareness, and easily digestable educational resources (how about SchoolOfData.org) need to be promoted.

Now then, about those success stories...  This list includes data publishing projects, groups and apps that have been built on Open Data.

• The Berlin Open Data Portal (100 datasets)
• Open Cities EU (various OD working groups)
• EuroCities (OD working group)
• OKFN OD working group
• Czech Republic Open Data Portal (171 datasets)
• Amsterdam Open Data Portal (137 datasets)
• Adopt A Hydrant (public upkeep of local area)
• Slovakia Open Data Portal (~52 datasets)
• Other version of Slovakia Open Data Portal.. (45 datasets)
• zNasichDani (links between businesses and Slovakia government)
• Cena Statu (Slovakia governement spending visualisations)
• OtvorenéZmluvy (Slovakia state profitability)
• Restart Slovensko (OD hack competition)
• Open Trip Planner (journey planning platform, deployed across 12 countries)
• FairPhone (pushing for ethically sourced smartphones)
• Locatable (where to live in London)
• Placr.mobi (public transport and venues in London)
• Mastodon C (green cloud computing)
• Apps for Amsterdam (OD hack competition)
• Apps For Democracy (Amsterdam gov't focussed OD hack)
• Kenya Open Data Portal (531 datasets(!))
• Farm Subsidy (EU spending on farm subsidies)
• Gapminder (Global data trends visalised)
• Europe Open Data Portal (>800 datasets)

That'll do for now.  Lots of the portals and competitions have links to app examples etc.  There's lots to explore.

Finally, I highlighted in my notes quite a lot of things that I need to find out more about.  A lot of them are technology or platforms for publishing or sharing Open Data, and various standards or studies I need to read in more detail.

I have a couple of questions to ponder on, too:

There's a massive focus around hacks (more often than not one off events) as a way of using and promoting Open Data.  What other ways are there?  What will the path to a deeper integration of Open Data in society look like?

There are lots of datasets and vocabularies about public services and society, as well as science and education.  What arts, culture and media datasets are out there?  (And what has been done with them?)  Ooh, or online social interactions?  Maybe I'll do a survey.

[1] Prof. Dr. Felix Sasaki, keynote: "Linked Open Data @ W3C-Vocabularies, Working Groups, Usage Scenarios."
[2] Prof. Dr. Simon M. Onywere, talk: "The Kenya Open Data Incubator Project – Outreach to Research Community."
[3] https://www.gov.uk/government/consultations/midata-2012-review-and-consultation via Nigel Shadbolt
[4] Dr. Rufus Pollock, keynote: "Open Data, Building the Ecosystem"
[5] Peter Hanečák, talk: "Open Data and Open Government Partnership in Slovakia."
[6] Michael Hörz, talk: "Open Data in Local Journalism: An Excel file?"

December 6th, 2012.

Notes as I scrawled them.  Read a proper review.  Purple is calls to action for myself.

Rufus Pollock - Open Data, Building the Ecosystem


Open API is a contradiction.  
An API is not open (though still valuable) but not the same as bulk open data.
Should we open governments?  So much is hidden.
Ultimately a lot of it is about in/justice (debts paid off by younger generations).

Open spending
- only 30% of departments are up to date - can keep checking who's updating their stuff.
- but sometimes more up to date that government's own records - they use it themselves.

Getting the data is only the beginning.  Need a platform.

Getting citizens, journalists and government to work together.

Why doesn't every dataset have a public issue tracker?  This is what he'd like to see most.

The process is important.

Digital doesn't run out.  That's why open makes so much sense.

FourSquare uses OSM.

Average consumers won't see raw open data - but via products and services.

Next?  Building communities.  Takes time to nurture.  publicdata.eu.

We'll go from hackdays to deep integration.  Toy vs. core datasets.

Geodata is mature.

SchoolOfData.org.  Data analysis, programming, training.
- Teachathons better than hackathons.

Should have access to all our own data - travel (Oyster), shopping (loyalty cards), social.
- Would like to control who uses it.
- Compare with others (aggregate).  Only companies can do that.
- Some people will choose to publish/open their own data.  Even if it's a few people, still lots of data.

Open Data = Platform; !Commodity
- Build on it rather than sell it (and let others build on it)
- Already seeing people building companies/making money on this principle.

Ecosystem.  Need feedback.  Poor quality data creates wasted time, especially at hacks.

Datagov census dashboard - who has what out.
There's no gov. data that's come back cleaned up from the community (even though people are almost certainly cleaning this data up).

Be patient.  Needed to wait for pressure to build up (why it's happening now).  Low investment.

Way to fund production of gov. data (or combination of):
- Charge users
- Taxpayer revenue
- Charge creators (most attractive)
  eg. Companies Register data; more efficient to charge people registering companies a bit extra, people won't be deterred from registering a company because of this.  Basically, add to existing fees for things people will pay anyway.

Open data saves lives!  Heart surgery data caused improvements.

MiData (more from Nigel)
Supermarkets not very good, but some big companies involved.

Prof. Esteve Almirall - Reinveting Cities - Open Innovation in the Public Sector

Competition now is about innovation, not money.

Adopt a Hydrant (Boston)

Peter Hanecak - Open Data in Slovakia

2012 govt signed open government partnership. 

Laws already align with Open Data principles. 

Everything must be public, unless stated otherwise (eg. according to a specific law, like army secrets. Must be clearly defined, no rubbish excuses for not publishing something). 

Open licences (GPL, CC) are not recognised, because there's no signed paper. Currently they're campaigning to fix this. 

Existing laws do not fully apply to regional governments, only to state government. 

Data that is available isn't in great formats. Lots of things are ignored or misunderstood.

Achievements so far: - Datastores and apps. Datanest.fair-play.sk, znasichdani.sk, cenastau.sme.sk, otvorenezmluvy.sk, data.gov.sk. - app competitions, conferences. Restart Slovensko.

Future

- Spread the word 

- Advance principles on regional levels 

- Major release of OD by gov 

- develop OS publication platform for OD 

- incorporate results into other projects

They have a standard published for state government.

Ivonne Jansen-Dings - Code4EU and Apps for Amsterdam

Technology is a key component, but impact on citizens is central. 

Taking linked OD out of academia and applying it to 'real life'.

Tourist one. FairPhone. 

Collaboration between government, coders and citizens. Can't do this top down. Creating something together, that has to evolve naturally. There isn't a formula.

apps for Amsterdam is a platform for people to talk about OD; creating it and working with it. 

• Lots of reasons why coders participate. 
• Hobbies
• Solve local problems
• Network, get new business
• Ideas start during hack events/workshops. 
• Past two years gone from 22 - 130+ datasets. 
• Quality of data is essential for good apps. 
• Mostly about getting the government to see the necessity of open data. 
• All sorts of working groups arising with municipalities to solve specific regional issues using OD. 
• Helping people who have made/are making apps, to help them move forward. 
• Lots of people are stuck because they need certain data. 
• Creating ideas for apps is a good place to start with deciding which data to open up. 
• Essential for app developers is also getting paid. Necessary to help people evolve apps. So a4A helps to connect developers to companies, NGOs, etc. for whom the app is relevant, so they can work out a way to work together.
• They don't have much government data about spending and stuff, working on that.

Apps for Democracy (Apps Voor Democratie) 

• Can see which parties and people collaborated with each other on getting motions put in and passed etc. Now integrated into actual parliament website. 
• PolitFutures. Stock exchange around politicians.

Code4EU

• Currently hiring developers to create solutions for problems they see within municipalities. 
• Changing government from within. Civic innovators.
• Lots of people come with problems, not ideas for apps.

Not about the apps, about the social change.

Local issues can have solutions that have global value.

Lena-Sophie Mueller - Open Government is more than Open Data

Stuttgart local people had issues with train station and there was a huge protest. Planning happened in a black box, people didn't understand why decisions had been made. 

(Ed trams)

Same as ACTA. People negotiated about it for 7 years, but it was not transparent.

Need transparency to face obstacles of 21st century. 

IT-Planungsrat say open govt needs to be a focus point, but will focus on open data first. 

• Open goverment though, is more than just open data. 
• It's hard to keep information secret now. Things get leaked. To counter leaks, they put it online too, so it's a trusted source. Can publish accurate new versions. 
• What's needed is a Diff, so information needs to be in machine-readable formats so documents can be compared. 
• Then politicians can be asked specific questions about changes.

Open gov means taking contribution of people as valuable. 

• US crowdsourcing patent information for making decisions about patents. 
• Open Budgets - people participate in deciding how budget is spent.
• Governments and administrations collaborate. 
• barnet.gov.uk. PledgeBank ("a site to get things done")

Lots of administrative workers aren't used to working with data. 

• Need help with extracting data and organisation and processes and technology. 
• Change management. 
• It's going to take some time, but it's worth making the journey.

Convince organisations that it's okay to open data (sans personal information). 

• Lots is digitalised already, and expensive to digitse paper. 
• Smaller municipalities are mostly paper. 
• eGovernment projects are important for open data.

Status of open government in a country? The thing Rufus mentioned. 

Initatives that measure freedom of information laws, but it's very difficult to measure openness. Need to develop a measurement that could be used internationally (big complicated project). 

Web Foundation published an index a few months ago. 

What does she use?  Usually use CKAN.

Christoph Lutz - Open Data and Social Media

Social medai readiness in Hamburg. 

Many insights from social media can be transferred to open data. 

Social media adoption and readiness can take place on several levels 

- organisational level 

- individual level 

- societal level

This project looks mainly at organisational and individual. 

Organisational: 

• different agencies, like culture and financial services; regional agencies. 
• structure, leadership and culture

Individual: 

• drivers and barriers to social media adoption: cognitive (know how) or affective (acceptingness of technology, concerns)

Some agencies in Hamburg have started intiatives (like fb, twitter). Some were recalled, but in general there is political support and Hamburg is more advanced than other cities. 

Their research project: 

• conduct interviews with employees who had contact with social media. 
• 8 people 
• analaysed successful vs unsuccessful social media use 
• different levels of responsibility, and different agencies 
• case studies 
• soon a big quantative survey

Organisational factors: 

• political support (crucial) 
• leadership support (some people high up in hierarchy aren't familiar with technologies) 
• autonomy and trust (people can experiment, be proactive) 
• structures of organisations (complicated, different motives and experiences in different departments, hard to coordinate) 
• processes (hierarchy and bureaucracy; you need fast feedback for social media, which contrasts with usual way of work) 
• resource (stressed most in interviews; employees don't have time at work to administer social media accounts, sometimes IT resources)

Individual factors: 

• age (younger people more interested) 
• affinity (how much people enjoy working with IT; intrinsic motivation) 
• experience 
• social capital 
• concerns (privacy, security, technostress)

Identifiable strategies: 

• avoid resistance (make projects appear small, non-invasive, simple) 
• externalise project (work with other organisations, avoid bureaucracy)

Engage people in open data via social media.

Objective of Hamburg project? Mainly about representing administration to the citizens, and providing feedback to people. Later about engaging people in conversations and participation.

How to make use of social media in administration? How to get public offices to produce open data? The quantitative questionnaire will help show how open people are to social media.

Simon M. Onywere - Outcome of the Kenya Open Data Consultative Forum - Kenya’s Strategy to Make Government Data available to Communities

Increase transparency and accountability of government. 

Help people make decisions. 

Support economic development in the country. 

Supported by World Bank.

In Kenyan Bill of Rights, citizens have a right to the data held by the government.

opendata.gov.ke 

Media plays a very important role.

KODI - Code4Kenya:

• Lack of certain data
• lack of metadata
• limited search
• data duplication. 
• Need for better analysis and visualisation tools. 

These observations allowed holding a stackeholders forum.

• People interested in solving problems that face the country.
• (MANY issues; things Europe was facing 100+ years ago, but no data about what's going wrong, what the impacts are likely to be). 
• Interested in building a platform, but the content is important, and what is it supposed to help us do? 
• Issues about data collection. 
• In the forum, ended up talking about issues that face the country, rather than the data.

Challenges: 

• data hugging syndrome: 'this data is mine' 
• Lack of Freedom of Information Act 
• Slow digitization 
• Lack of trust and low culture of openness

Demand for open repository of all Kenyan PhD and Masters theses. Needs metadata, and needs to be widely accessible. Will help with research, and help people understand research that has been done that might be able to solve problems. Help avoid double research.

Not really free information, because most public sectors spend money to get information.

Project requires various degrees of collaboration. Countries in Europe that are one or two steps ahead. Need training for socio-economic transformation.

Are the licensing issues being sorted out? There's a comprehensive statement on the website (so a custom license?). 

Citizens will hold government accountable for information provided, so they (gov) are worried about data being inaccurate. 

Letting research community use the data can help get useful feedback about what is wrong - you don't know if things are wrong until you try to use the data. 

Up to people to give the government the correct situation on the ground, because gov't is not always right.

Nigel Shadbolt - Finding the Value in Open Data

Local data matters. 

Open Data Institute - build economic value in a serious way. 

2005 AKTive PSI was early beginnings with getting various bodies imagining opening their data. Nobody would really give them the data at that point, but were curious about what could happen. 

Reported to parliament in 2007 - said it was exciting and had great promise, but nothing else. 

Activism, top-level political will and committed individuals needed to move things on.

Two years is not a long time for really disruptive movements like open data. Natural lifecycle to processes. 

Early enthusiasm, but wall of people who question impact, value, if it's worth the effort. We can see now that it's always worth the effort, and doesn't cost much.

Government is becoming more comfortable with this stuff. 

"Can we put a government website up that says 'beta' in the right hand corner, and not be ridiculed?" 

Gov't got used to the idea of agile web development; of not knowing how the system would be finished when it's started.

Suddenly, applications appear. Many unexpected. 

Gov't efforts would be more expensive and less effective.

Virtuous cycle: open licences - open standards - open source - open data - open participation.

CC isn't for everyone... companies worried about giving rights away. In the UK there's an open government license, developed by government lawyers (to make officials feel more comfortable, and understand conditions).

Open Data market needs a steady stream of successes. Always have a story the person in charge can understand. 

OD is abstract principle. 

He uses MRSA. They began to publish infection rates in hospitals. Two years later, it's down by 85%. Worst hospitals can look at ones that are doing better. People started to ask questions; simple procedures implemented (sunlight, disinfectant).

We have to understand that companies are in the business of making money, and public services are in the business of providing efficient services.

Transport... Companies think it's more valuable to keep hold of the data than to have people on the transport - lolwat?

Visualising data highlights issues.

Quality over quantity. 

Routine to publish certain sorts of information, but there's other stuff that's really important. 

Needs to be found easily. 

Data portal needs lots of metadata (quality of content, what kind of links, how much). 

Every public data should rate itself on the 5* score card.

Open Data business models: 

It isn't enough just to publish. 

Need to build demand for data that you're supplying. 

If the data is poor or gets turned off, people will let you know. 

Data Marketplace. 

OD Apps (people think this naturally). 

Innovate - economic benefits for host, sponsor and developers. 

Developers innovate on behalf of companies. 

Build and maintain trust. 

Prove that you're doing good things, eg. where materials come from, or effects on environment. Various different kinds of open APIs.

Open Data needs a balanced and broad ecosystem. 

Complex. 

Not just gov'ts. Businesses are beginning to, citizens might eventually (think about social networks). 

Lots of varieties of open data AND closed data (some just cannot be released). Or personal data that only the owner can/should have access to. 

It's much richer than just "everything's open and we need to work out a way to monetise it".

midata: 

We will increasingly become aware that we can collect our own data. 

So why can't we get the data that other people collect about us? Why don't you have access electronically to every receipt - and what would that world look like? Switching suppliers, teaming up with neighbors for shopping. 

Energy providers in the UK (three of them, other three will do soon) give access to all their raw data.

It's hard to get data out of companies, but those who do see real benefits. Telephone companies in the UK are seeing increasing data exchange between them and consumers.

Products and processes that we'll see emerge from this are exciting.

What's the mix between open data and personal data (midata)? 

Government midata. Most people don't claim cold weather allowance. Costs a lot to get credits moved around. Open data meeting midata would benefit this. Same in health area.

Open Data Institude - theodi.org 

Leading the creation of the open data ecosystem. 

Trying to improve public supply. 

Training people to produce and publish open data. 

Incubating companies. 

Work with public bodies, big corporates, small startups, trying to find values in datasets. 

After 8 weeks, 4 companies working in their space. 

Locatable

placr.mobi is a transport API provider. High quality access to all open transport data. 

Mastodon - green cloud computing options. 

It isn't all sweetness and light. Have to demonstrate tangiable benefits to keep progress going. Have to give company/politicians good reasons why this is better.

Huge amount of capital value is based on "I know something you don't know". 

As information becomes abundant, the landscape will be changed. You can't rely on knowing something any more, you have to provide something extra quality. 

Drive innovation and improvement in service delivery. 

With heavy investment in acquiring certain information, why should they share it? 

Evolutionary arms race means that someone else will find a way to collect that data more cheaply. How long can they sit on their monopoly?

Governments are not here to become revenue generating businesses, but to provide public services.

In the UK, the office of national statistics gross value added figures are from 2010.

Hasn't seen any examples yet that cost more than the benefit that's gained. 

Hospitals, traffic data. These studies need doing more carefully than they have.

December 5th, 2012.

Notes as I scrawled them.  See here for a proper review.  Purple is calls to action for myself.

Dr. Philipp Mueller - Openness as a means, not an end
I missed the opening keynote.  His slides are here.

Dr. Wolfgang Both - One Year Open Data Portal Berlin

Open cities EU - Nov '10 - Apr '13.
Amsterdam, Barcelona, Berlin, Helsinki, Paris, Rome.
Berlin responsible for open data working group; several working groups (OD is just one).

EuroCities

Knowledge Society Open Data working group
Guidebook for cities, 30 pages so far.

Open Data Berlin
Portal Sept '11
Publication
Press conference Feb '12
Short term: Political agenda, budget, working group.
Mid term: Harmonize data formats.
Long term: Legal framework (Berlin can't decide laws by itself, for whole EU).

Open Data Day May '11, '12 and '13 in prep.

WG open traffic hack (29th Nov '12)
- 150 programmers with transport data.

Portal stats
Are users interested?  Peak at start.  Other peaks for hacks, Apps4D contest (Nov '11)
Possibility for feedback - questions, advice, ideas.
100 datasets; daten.berlin.de

WG 2012
- formats and metadata
- licensing and user rules
- education for staff (lectures, this is new for many working in public sector)
- organisting and processing
WG 2013
- evaluation of OD studies
- Recommendations
- Exchange with other cities.

(Q&A)
Datasets were volunteered, not selected
- But are looked at for quality, machine readable, looking for wide range of topics of interest to public.
- Want open, transparent process for publishing.
- Communicate with media as well as community.

Licenses
- Heuristic... no legal advice available because it hasn't been done before.  Many possibilities; for opening data for individuals, CC was familiar to Internet community, includes origin data (CC-BY).  Some smaller datasets are licensed for non-commercial usage.  Discussion still ongoing.
Knows other cities will follow / copy there example whatever they do!

Jan Schallabok - Right to Freedom of Information on Enterprises


Call for open enterprise data.

Why?
Scenarios, set a timeline:
2015 - personal search
2016 - data disasters (identity hack)
2017 - pictures omnipresent, know all about everyone becomes normal
2019 - Google Glass on market

If there's no data on you, things don't work (eg. personalised advertising)
Society down the drain if it didn't open data (Switzerland in the story didn't open data, so Swiss woman moving to German couldn't settle in easily).

Moving away from clear facts towards probabalistic.
eg.
Google Translate fed by open (input) data, but algorithms aren't open.
Siri - enriches dataset from Web (he said Google search?)
OpenStreetMap (counter example)
If there was more data, everyone would use OSM paradigm (eg. government).

Harm businesses?
..maybe.  But more damage in the long run?

Need to make businesses move away from using peoples' data.   Like, we work for facebook.  our data, not theirs.

Data protection:
by law, data subject has rights to know logic involved
but
as long as it doesn't affect trade secrets

Privacy implications
'Anonymous' data can be used to identify people.  See AOL search database fail.
When can datasets go public?
Weather data can be personal data (no time for example..)

Michael Hörz - Open Data in Local Journalism

Journalists expect everything

- spending

- political decisions

  (district levels, searchable)

- quality (schools, pollution, food)

- real time sensors (air quality, traffic, energy)

- Open Data Paris (loads, on a map)

- Locrating (school performance on map, UK)

- Chicago bike crash reports (map) (sort by injury, date, day; data all open from Chicago Transport Authority, in a nice format).

- LA Times LAFD (fire dept.) response times.

- Airplane noise map, taz.de (Journalists had a PDF, eugh).  One of the first Berlin interactive visualisations.

- Berlin election.  Was real time.  Down to the polling station.

- Berlin bicycle accidents '11 - came from massive PDF (3,800+ cases)

- Wishes for xls or csv... wants directly processable.  WHY NOT RDF?!

- ..or APIS.  WHY NOT LD?!

Reality = PDFs, requests ignored, data incomplete or hidden.  Hard to get for journalists.

All datasets are interesting and should be out there.  In Berlin often only one or two districts are available, which is no good.

(Q&A)

It's not always straightforward just to release data - need priorities; raw data/API documentation isn't always available straight away.

Why PDFs?  They don't know any better.  Need to make people aware.

Consequences of public seeing data they're not used to?  Panic?  Or activism?  Pressure politicians for change.  Empowers people.

Is there are resistance to making data available (eg. Italy - data there but useless).  Maybe, or maybe they just don't realise [it's useless].

Prof. Felix Sasaki - Linked Open Data @ W3C-Vocabularies, Working Groups, Usage Scenarios

== first half of MASWS. 

New work on LD http://www.w3.org/2012/ldp/wiki/Main_Page 

Media fragments - spec finalised www.w3c.org/TR/media-frags/ 

Ontology for Media Resources (DC for video and audio?) http://www.w3.org/TR/mediaont-10/ 

Internationalization Tag Set 2.0

SW core is stable, so work with vocabs now. Need interoperability. Decide: - Syntax - - Microdata not necessarily for SEO - - Schema.org; w3.org/wiki/WebSchemas; very basic schemas with increasing numbers of more specialised extensions. Discussion at lists.w3c.org/Archives/Public/public-schemas - 

Application scenarios.

Organisation ontology - Membership and reporting structure, location information, organisational history - Interoperable organisations - 'Final call' stage - nearly done. Need feedback.

DCAT (interoperability between data catalogues) - Uses FOAF, DC, SKOS

w3.org/TR/publishing-linking draft Namespace neutrality - xmlns.com

Language graph of the Web is cool.

Tomáš Knap - Tracking Data Provenance of the Published (Linked) Open Data

xrg.cz

opendata.cz

watch film

Defines provenance and agents, artifacts, processes. Provenance useful for data integration. Which is right/recent etc. 

How to cite. Vocabularies: PROV-O (almost w3c final, w3/ns/prov), VoiD (datasets w3/TR/void), FOAF, DC ODCleanStore - http://iswc2012.semanticweb.org/sites/default/files/paper_37.pdf (prov aware storage, processing, querying) - Write rules/queries using web front end. Certain automation from inserting ontologies. Still manual work.

LOD2 WP9a - EU project LD tools

Maria Magdalena Theisen - Open Data and Big Data

Big Data - have to ask questions to understand what questions to ask. Consists of volume, velocity, variety. Can't say that all open data is big data, and vice versa.

Some BD from external social media, disaster information, sensors, smart meter. Lots of things bringing data to process.

Facebook has largest data collection by 2010.

Open and Big - Eye on Earth - air watch (over 1k stations in Europe providing live data), noise watch, water watch (static historical data) - can rate quality of data and give attributes

Cloud computing is an enabler for B and OD - Don't have to manage servers to provide data. - Flexibility and scalability - Interoperability with existing infrastructure - Easy access to data - Development platform (Azure) - can enter an app with open data into marketplace. Lots of examples.

Is Azure marketplace integrated with ckan? - No.

Evanela Lapi - Building Sustainable Open Data Platforms

Understand stakeholders

* Consumers

* 

* Developers

* Citizens

* 

* Less technical, can use open data to help with life

* Journalists, scientists, researchers

* First two more critical

* 

* disseminate data

* Need open, standards-based, non-proprietary formats.  Easy to download/browse/search/redistribute/share.

* Publishers

* 

* Provide transparency

* Want a cost-effective, easy solution platform

* Public sector has lots of data not online - because it's hard to publish?

* 

* lots of friction, fragmentation

Socrata - end to end, custom solution.  Many implementations in US, Kenya.

vs.

Integrated, loosely-coupled - existing SW, eg. CKAN + Drupal (data.gov.uk)

Faunhofer OD platform is Java (Amsterdam uses)

Open Cities

- open innovation (see last time this was mentioned)

- on Github - get feedback from use.

Virtuoso triplestore + Liferay CMS + CKAN catalogue

(Java wrappers for REST APIs)

User roles:

• Data owner
• Publish
• Maintain
• Bulk upload
• Platform user
• Query
• Discuss
• Search
• Browse
• Download
• Propose new

amsterdamopendata.nl - 137 datasets in 18 categories

flevoland.....nl(?) - 22 datasets

It's a good start, but still not enough - why?

• Too much manual work, redundancy across different platforms.
• Modernise environment - by modular, high level stuff?  (I think that's what she said)

"Germany isn't much into OD yet.."

Oliver Adamczak - Big Data for Smarter Cities

Leaders must innovate to exceed citizen expectations.

Functionality of BD - use variety and volume to innovate.

Vision - do things you haven't thought of before.

I should do a survey of open data about arts/media?  It's all about gov/science.

IBM BD platform

Hadoop to store

- low cost (open source)

- scaleable

- easy to load data - don't have to care about structure until afterwards

Text analytics to read PDFs etc. and extract data with context.

Streaming data is important

- Not for repo, just use/analyse and discard.

3rd December - 9th December

I went to the 1st International Open Data Dialogue (#odd12) in Berlin on Wednesday and Thursday, and gave a lightning talk at Digital Methods as Mainstream Methodology (#dmmm2) in London on Friday.

Both were brilliant; I met a ton of interesting people, learned loads and gained much inspiration.  My extensive notes are in the process of being typed up and thought about, and will be published on here as soon as humanly possible!

Now I'm going to catch up on sleep that crack-of-dawn flights and many hours of train journeys have denied me recently.

19th November - 25th November

I read two papers about ontology development methodologies.

• Notes are here.

I read two articles by Bennett Haselton about decentralized social networking, which happened to pretty much sum up and beautifully articulate everything about that that has been floating in my subconscious for a couple of weeks.  I saw links to them in the latest Circumventor email, which I've been subscribed to since High School for bypassing the internal blacklist, and remain subscribed to because the jokes at the end are always laugh-out-loud funny.

I attended an all day course entitled 'practical project management for research students'.

• It was attended by a diverse bunch of seemingly really lovely people.
• The two ladies running it, from the IT Project Management department in the University, were lovely too.
• The stuff covered was all obvious, common sense stuff (and pleasantly the organisers didn't try to claim otherwise) - but sometimes it's helpful to have it all written down and waved in your face.  And structured, in particular.  Made me actually focus on thinking about organising my project.  The main thing I hadn't much considered, even subconsciously, was formally identifying stakeholders for a project and their relative interest/power in the project.
• There are a bunch of tools at projects.ed.ac.uk to aid in project management.
• It prompted me to do these week-in-review posts, as I realised I haven't been recording properly everything I've been doing (an overview of my time goes on my calendar, but no detail).
• The sandwiches weren't great, but fortunately when I got back to the Forum there were massive slabs of chocolate cake left over from some event.  I love the Forum.

I booked a place at the 1st International Open Data Dialogue in Berlin, and necessary flights.

• Despite the short notice, it worked out logistically because I need to be in London on the 7th anyway, so I can simply go to London on the 4th, fly to Berlin from there for the 5th and 6th, and back to London for the 7th.
• I'm particularly looking forward to "Open Statecraft: Openness as a Means (not an End)" by Philipp Müller, "The Open Data Movement vs. Business Models - is this a Contradiction?" by Dr. Peter A. Hecker, "Linked Open Data @ W3C-Vocabularies, Working Groups, Usage Scenarios" by Prof. Felix Sasaki, "The potential of Open Data for improving urban sustainability" by Dr. Marianne Linde and "Towards Trustworthiness: Establishing Transparency with Open Information Flows" by Dr. Edzard Höfig.
• I'm also looking forward being in Berlin again, even if it is just for one evening, and I'll probably be too exhausted to appreciate it.

Ontologies with a View took place at a different place and time to usual.

• And with added pancakes.

I started preparing for presenting at Digital Methods as a Mainstream Methodology in London in a couple of weeks.

• I scribbled lots of notes.
• I skimmed a few papers by organisers/speakers but didn't read any in detail yet.  Mostly stuff about analysing data gathered from comments, tweets etc. 
• There will be more about both of those things next week, I imagine.

I made a plan for the two weeks following the 20th.

• It mostly consists of finishing my Digital Methods preparation.  I have a lot of non-PhD related things to do as well, plus lots of travelling.  Also graduation from my MSc, and subsequent parental visitation will get in the way.