I had two meetings with Dave Robertson, my second supervisor, about what on earth I'm doing, and here is a vague summary of my thoughts afterwards.

I came to the realisation between meetings that I need to scrap the term Amateur Creative Digital Content, because amateur doesn't really apply by its true definition and creative is too subjective anyway.

Focus on content creators, not content (so previous point doesn't matter so much anyway; maybe just need to look at existing ways people are describing types of users to make it clear who I'm concentrating on).

In terms of emphasis of the thesis, I need to make a choice between taking a cognitive science/sociology perspective and a tecchie/engineering perspective (I choose tech because that's where I'm most comfortable, but the sociology side of things is still important).

(Therefore) I need to think concretely now about technology architecture.

Not to get too hung up on the Semantic Web; the technologies are a vehicle for testing theories, rather than an end in itself (though I still think facilitating a big linked data set of this sort of data is useful in the long run for research and practical applications, I didn't labour that point).

Social machines, and how Dave's process modelling language fits in, which I think I get in theory but not practice (I'd probably have to look at a working application and code to understand really). Some of the principles may be useful further down the line, but probably not the language itself or anything.

Technology-wise, I'm not thinking about anything novel or new, but more new ways for how various Web and SW technologies are combined and applied to this domain. (?)

So maybe the novelty is in marking up various things about content creators and using this to infer information about the processes they're involved in (or want to be involved in) in order to then facilitate these processes, without (necessarily) ever explicitly representing these processes (because from the content creators' perspective, they're certainly not thinking in terms of formal representations of processes, and in many cases won't know what they're trying to make until it's done, for example).

How to represent the inferences made might be novel and exciting, but I don't know.

Hmm, I still don't think I've figured out how to evaluate .. anything. Beyond comparing activities of users with magical-new-system vs without magical-new-system. And maybe, going back to the this-big-dataset-is-useful idea, by finding questions we can now ask about these kinds of communities that we couldn't before because they were so fragmented.

For easy navigation of my gibbering.

First impressions of Madrid (6th and 7th, #travel)

Mon 8th: Research in theory and practice, and where on earth am I?

• Notes for keynote by Frank van Harmelen (questions we couldn't have asked 10 years ago)
• Notes for Introduction to Linked Data by Mathieu D'Aquin
• Notes from invited talk by Manfred Hauswirth (linked data streams)
• Notes from poster session

Tue 9th: Collaborative ontology engineering and team formation

• Notes from Lynda Hardman (telling stories with media)
• Notes from Natasha Noy (ontologies and Protégé)

Wed 10th: Practical semantics, and human nature.

• Notes from Harith Alani (Social Media Analytics with a pinch of semantics)
• Notes from Tommaso Di Noia (recommender systems; tools and techniques)

Thu 11th: Social semantics and serendipity

• Notes from Fabio Ciravegna (situation awareness via social and physical sensing)

Everything from Friday 12th July

• Notes from Peter Mika (Yahoo!; semantic search)
• Notes from Raphael Troncy (semantics at the multimedia fragment level)
• Notes about our mini-project (Evaluating Serendipity)
• Summer School for Semantic Wizards (video)
• A cruel joke, a great party and a late night.

Final impressions of Madrid (14th, #travel)

Make a model of what is happening.

WeSenseIt - citizen water observations.

River belongs to citizens, not authorities.
Physical sensors (hard layer) are expensive and brittle.
So use people instead (soft layer, social).

Give people small sensors.  Phones.
Then you just need software for information management.

• capture.
• integrate and correlate data.
• share.

Can't rely on phones.
Old people in Doncaster.

Give them easy sensors instead.

• camera
• humidity
• position GPS
• water depth, velocity
• rainfall via accelerometer
• could coverage via luminocity

Costs about EUR 80.

Open Source & hackable.

Not expected to substitute professional sensors, but a way to crowdsource information you would never get.


In Delft

Give people flood preparation advice and record who ticks things off, to build a picture of who/how/when preparations take place.


The Floow Ltd

"Commercialises data solution for telematic insurance."

World divided 10x10m squares, sense things everywhere.
Traffic risks.


Sensors tell you people are going somewhere, not why.
That's what social media can tell you.



Monitoring development of a house fire via Twitter.
Seeing events through the eyes of the community.

Social streams:

• High volume
• Duplicated, incomplete, imprecise, incorrect
• Time sensitive / short term
• Informal
• Only 140 characters
• Spam

Large music festival.  Monitor geolocated messages, trends, topics and relations.

Most 'critical' events were management issues.
Developing system to warn you automatically about things to pay attention to.

Look/listen for event within 72 hours.  10 minutes to find out what it was.
- Simulation of station bombing.
Minute by minute description of event.
1.5 billion messages.

• Linguistic issues
• Alternative language
• Negatives
• Conditional statements
• Hope/prayer statements
• Irony/sarcasm
• Ambiguity
• Unreliable capitalisation
• Data sparsity

Four things when monitoring:

• What
• Identify, classify, cluster
• Events and sub-events
• Involved entities
• Who
• Human or not?
• Bots can be beneign, but many are a serious risk.
• Bots that pretend to be humans.
• When
• Where

Big problem - people tweet crap!
People don't realise when people nearby are in danger.


Deception on social media

False crowdsourcing political support on social networks.
Smear campaigns using bots.
Bots to foster / prevent social unrest.


Identifying bots

23 behavioural features.
Feature set is open.
Recognise 90% of bots - more than humans can do.



Very small amount of tweets are geolocated, it's useless.
Have to use the text.

Timestamp is not necessarily correct.


Issues in events

No infrastructure (eg. at music festivals).
Phone signal issues, phone charging issues.

Most tweets from outside event.

Conclusions

Need to convince citizens that authorities are not spying on them.
Need to convince authorities that citizens are not all criminals.

Privacy and legality issues.

Creating a company on this research would be unethical.
Need to pass the right message.  Full disclosure.  Non-intrusive use of tweet content.

What happens when authorities demand this technology for privacy-invading stuff.

Have to be careful with what you publish.
Always assume the bad guys have thought of what you thought of.
Always be in a situation where you can destroy your data at short notice.
Bit legal barrage behind them.  Know what they are/aren't allowed, know what they do/don't have to do.
Start leading a blameless life.

Tools and Techniques

Recommender systems

Input: Set of users + set of items + rating matrix.
Problem - given user, predict rating for an item.

In real world, recommendation matrix data is sparse.

Can use hybrid approaches.

Collaborative RS:

• Like Amazon.
• Based on other users with similar profiles.
• Experimentally better than content-based, but you don't always have many users.

Knowledge-based RS:

• No/little user history.
• Based on domain knowledge.

User-based collaborative recommendation:

• Pearson's correlation coefficient - baseline.
• Imagine millions of users - computing similarities takes a lot of time.
• So ..

Item-based collaborative recommendation:

• Focus on items not users.
• Compute similarity between each pair of items.
• Don't have to compute similarity between items that don't have overlapping ratings.
• Cosine similarity / adjusted cosine similarity (taking into account average rating related to a user to eliminate some bias).

Content-based RS:

• Based on description of item 
• and profile of user interests.

• Items are described in terms of attributes/features.
• Finite set of values associated with features.
• Item representation is a vector.
• Don't necessarily have complete descriptions of items - just have a 0 in your vector.

• Similarity between items: 
• Jaccard similarity.
• Cosine similarity and TF-IDF (term frequency - inverse document frequency).
• Batch compute similarities offline, then use similarities to compute ratings on the fly based on user profile.

• Predict rate only for N nearest neighbours of items in user profile, that are not in the user profile.
• An item is worth rating if more than x of N number of neighbours are within user profile.

Using LOD

To mitigate lack of information/descriptions about concepts/entities.

Recommender systems are usually vertical, but LD lets you easily build a multi-domain recommender system.

To avoid noisy data, you have to filter it before feeding your RS.

Freebase.


Tiapolo

• Automating typing of DBPedia entities.

Vector space model for LOD

• MATHS.

Collaborative ontology development from Natasha Noy (sssw2012)

Stanford, Protégé.

In past 10-15 years, through collaboration with scientists (particularly biomed), ontologies have become essential.

Don't need to sell ontologies to scientists, they believe in it.

Focus on science because that's where she has experience etc.

We're not so bad at versioning ontologies, more versioning data is the problem.

Experts add stuff, curator checks quality, and publishes upcoming tasks.

Similar to open source developments, but no research to compare the two.
- Different because biomed people are paid (well).

ICD - International Classification of Diseases.

• Started 17th century.
• Causes of death, medical bills, policy making.
• Revised in 80s over 8 annual conferences.
• 17-58 countries, 1-5 person delegations, mainly health statisticians.
• Manual, on paper.
• Whoever shouted loudest..
• Paper copies, only English, pdf.
• ICD-11 - OWL ontology!
• Open, Protégé (a customised, Web version), links to others.

Conflict resolution:

• People naturally don't step on each others' toes.
• Users expect stuff like Web 2.0 interactions, Web interface.

Web Protégé:

• No consistency checking - coming but currently must go offline.
• Ontologies are solution to everything - versioning, roles, social interactions.
• Also plugins are the solutions to everything - visualisations.

RELEVANT.

Users (consumers?):

• Finding content
• Media types * mostly text at the moment, little integration of different types
• Specific tasks - not much connection of results with user tasks.

More data than just what you seen in the media (cue my Venn diagram).

Plus, eg. paintings - lots of 'cultural baggage'.

Care more about the story than the media.
Interpretation by end users.  Hopefully message that the author intended.

Meaning of combination of assets.
eg. Exhibition of artists work.

Interacting further with the media.

• Search - serendipitous or focussed around a theme (or both).  Different search goals.
• Sharing, passing it on.

(SW and multimedia community need to work together).

-> Raphael Troncy on Friday - attaching semantics to multimedia on the Web.

Need mechanisms:

• to identify (parts of) media assets.
• associate metadata with a fragment.
• agree on meaning of metadata.
• enable meaningful structures to be composed, identified and annotated.

Workflow for multimedia applications

• Canonical processes of media production
• Reduced to the simplest form possible without loss of generality.

Heard of MPEG-7? Don't bother.. very much from a media algorithms perspective.

Applications:

• Feature extraction.
• News production.
• New media art.
• An interactive exhibit that responded to audience present.
• Hyper-video.
• Linked video.
• Photo book production (CeWe).
• (Using this example for explaining processes).
• Ambient multimedia systems with complex sensory networks.

Canonical processes overview...

There's a paper.

CeWe photobook - automatic selection, sorting and ordering of photos.
Context (timestamp, tags) analysis and content (colours, edges) analysis.

Things from these you want to represent your digital system (ie with LOD):

• Premediate, eg.
• remember to take your camera on holiday.
• write scripts, plan shots.
• place a security camera in the right location.
• Construct Message (not really in the chain, appears all over the place); what to conveny with media? Intention? eg.
• show people a great holiday.
• sell a product.
• inform/advise.
• Create (method of creation might be important, so record in metadata), eg.
• take photos.
• make video.
• Annotate, eg.
• automatic or manual.  Stuff that is embedded by device vendors (but there's so much more...)
• domain annotations: landscapes/portraits, timestamps, face recognition.
• Publish, eg.
• compose images into photobook.
• Distribute, eg.
• print photo book and post.
• cyclic processes online.

COMM - Core Ontology for Multimedia.

Premediate and construct message - human parts, she doesn't expect them to be digitised any time soon.

Using Semantics to create stories with media

Can we link media assets to existing linked data and use this to improve presentation?

How can annotations help?

• What can be expressed explicitly?
• Message (somewhere between a html page and poetry).
• Objects depicted.
• Domain information. <--- li="">
• Human communicaiton roles (discourse). <--- li="">

Vox Populi (PhD project)

Traditionally video documentary is a set of shots decided by director/editor.
vs.
Annotating video material and showing what the user asks to see.

interviewwithamerica.com

Annotations for these documentary clips:

• Rhetorical statement; argumentation model (documentary techniques).
• Descriptive (which questions asked, interviewee, filmic).
• Filmic: continuity like camera movements, framing, direction of speaker, lighting, sound - rules that film directors know.
• Statement encoding (eg. summary what the interviewee said):
• subject - modifier - object statements.
• Thesauri for terms.
• Can make a statement graph, finding which statements contradict and which agree.
• (He encoded this stuff by hand - automated techniques aren't good enough).
• Argumentation model - claims, concessions, contradictions, support.

Automatically generated coherant story.

• Are we more forgiving watching video? (Than reading these statements as text).  Peoples' own interpretations strongly affect understanding of the message.

Vox Populi has (not for human consumption) GUI for querying annotated video content.

User can determine subject and bias of presentation.
Documentary maker can just add in new videos and new annotations to easily generate new sequence options.


User informatio needs - Ana Carina Palumbo

Linked TV.  Enhancing experience of watching TV.  What users need to make decisions / inform opinions.

• Expert interviews (governance, broadcast).
• User interviews - what people thought they need (215 ppts).
• User experiments - what people actually need.

Experiment - oil worth the risk?

• eg. people wanted factual information from independent sources; what the benefits are; community scale information.

Published at EuroITV.

Conclusions

• We can give useful annotations to media access, useful at different stages of interactive access (not just search).
• Clarify intended message. Explicity with annotations.
• Manual or automatic.
• Media content and annotations can be passed among systems.
• No community agreement in how to do this. <--- li="">
• How to store?

Questions

Hand annotations are error prone - how to validate?
Media stuff - there can be uncertainty, people don't always care.

Motivating researchers to annotate...
Make a game.

Store whole video or segements?
W3C fragment identification standards - timestamps via URLs.

Things to investigate further!

LibRDF - linkeddata-perl for Debian by Kjetil Kjernsmo.

Rakebul Hasan, Fabien Geandon (? not sure about names, can't read my handwriting..) - Trustworthiness of inferences.

Taldea - fostering spontaneous communities.
Ghada Ben Nejma.

NERD ontology for spotting entities.
nerd.eurecom.fr
See photo:

It's a dynamic world! Ubiquitous streams and the Linked Data Web from Manfred Hauswirth (pooh42)

Streams: Any time dependant data / changes over time.

Has done a paper about P2P stuff.

Data silo - "natural enemy of SW scientists"

Massive exponential growth of global data.

Still have to integrate dynamic data with static data.
Multiway joins are domintion operator.  Need to be efficient.

Everything/body is a sensor.

Various research challenges:

• Query framework.
• Efficient evaluation algorithm.
• Optimise queries.
• Organisation of data.

CoAP ~= http for sensors.

Stuff about sensor networks and context - useful for Michael.

• Common abstraction levels for understanding.
• SSN-XG ontology
• Application: SPITFIRE
• You can buy a sensor off the shelf that runs a binary RDF store and can be queried.  So possible to use SW tech with resource constrained devices.
• RESTful sensor interfaces stuff being standardised - CoRE, CoAP.
• Linked Stream Model
• CQELS-QL (extension to SPARQL 1.1; already legacy)

Rewrite query to spit out static and dynamic - lots of overhead.
But need to optimise between these.
Neither existing stream processing systems nor existing databases could be efficient enough.
So the built own LD stream processing system.  (Optimised and adopted existing database stuff).

HyperWave - didn't succeed.  Didn't listen to customers and wasn't open source (license fees).
But better than hypertext was back in the day.
Performance important for success/uptake.

Just putting it on cloud infrastructure doesn't mean it scales.

• Need to parallelize algorithm.
• Took it to a point where adding more hardware did help.
• Problems!  Inconsistent results, engines don't support all query patterns.. very early, don't fully understand yet.
• Long way to go.  How to prove what is a correct result?
• Needs to be easy to use - dumb it down.
• Linked Stream Middleware (available):
• Flights, _live trains_ - SPARQL endpoint!, traffic cams.
• SuperStreamCollider.org
• Current Tomcat problem with twitter streams.

To do?

• Scaleability
• Stream reasoning (only processing, pattern matching, so far.  Want to infer conclusions).

World is:
... uncertain, fuzzy, contradictory.
So combine statistics and logics.
Hard to scale logical reasoning, so use statistics to shoot in the right direction.

Privacy?

• Build systems! Can't do thought experiments about the Web.

Don't get hung up on approaches / labels.

(by Mathieu D'Aquin).

Introduction to Linked Data from Mathieu d'Aquin

Linked Data = universal connections, like Lego.

Universal is why it's important.

Workshop instructions.

The only problem we had during the workshop was disagreement about how to read the 'broader' and 'narrower' relations between courses.  It instinctively ready contrary to what (my) common sense suggested (eg. that 'arts and humanities' is broader than 'history', which some people disagreed with).  A quick reference to the ontology documentation resolved that.

Semantic Web questions we couldn't ask 10 years ago from Frank Van Harmelen

Semantic Web & Web of data = a more manageable mission.
Metaweb movie - got bought by Google and incorporated into Knowledge Graph.

SW Principles:
1. Give everything a name (entities).
2. Relations form graph between things.
3. Names are addresses on the Web (so we inherit properties of Web like AAA).

This becomes Giant Global Graph.  (Maybe SW should be called Giant Global Graph?)

4. Add semantics.

• Types of things, relationships.
• Hierarchy, constraints:
• Inferences.  Bounding shared beliefs by sharing ontological information.  Space for confusion gets smaller and we begin to agree on interpretation of information.

Semantics = predictable inference.

Google: from just links to results, to information boxes (last May).  Can't directly address Google Knowledge Graph.
NXP (microprocessors): 26,000 products. Integrated all databases into triplestore.  Exposing subset of triplestore to customers.
BBC: 125 million triples.  Many data sources.  APIs to website.  Own ontologies.

All have the same triple-layer architecture:

Raw data
   |
SW layer
   |
Output / API / UI etc

DataGov: eg. air quality in cities, campaign money, if policies work.

Companies don't care about SW, but are using these technologies for their own IRL purposes.

These are all different types of use cases of SW technologies:

• search;
• data integration;
• content re-use;
• SEO;
• data publishing.

It's important that the SW graph is so big.

• More questions to ask.
• Good that we no longer know how big, or how fast it is growing... Tens of billions of facts.
• How many are really permanent?
• Some are stable, some will disappear - just like the 'regular' Web.
• "...it being a mess is the only reason why it scales."

We need to get used to the idea of SW being a mess - aka "a system so large you can no longer enforce central control" (complex system).

The LD cloud is still poorly interconnected, but good graph properties.

SameAs.org

Heterogeneity is unavoidable.
Socio-economic, first to market - why certain systems/ontologies get used, eg. schema.org, dbpedia.

Self-organisation.
LD cloud grew, nobody designed it.
Knowledge follows power curve.  This has an impact on mapping and reasoning, storage and indexing.

Distribution.
Web not geared for distributed SPARQL queries.  Everyone pulls in all data and queries local copy.  Not very 'webby', disadvantageous.  So subgraphs?  Query planning?  Caching?  Payload priority?

Provenance.
Representation, (re)construction.  Metametadata (knowledge about knowledge; uncertainty; problems with vocabs for this).
How to get from provenance to trust.

Dynamics (change).
Cool Web in 60 seconds graphic.
SW not changing this fast, but soon..

Errors and noise.
Sometimes we disagree.
Deal with by: avoid, repair or contain.  Or just deal with it - allow argumentation.
Fuzzy, rough semantics - almost, maybe.

Lots of research questions.  But not ones we could ask 10 years ago.

Information universe - "algorithms exist without us looking at them".

We should ask if things work in theory.
Scientists vs. engineers.
Discovering vs. building.

• Is this incidental or universal?

OWL is our microscope.
We can see structure well in some domains, but not so well in others.  Maybe it's our tool that distorts, rather than a property of the domain.

Says we should change our mindset from building stuff to hypothesising and falsifying.

Just notes from a three-hour workshop about how to write an Informatics thesis, on the 16th of April.


State contributions (to knowledge) explicitly.  Intro, conclusions; each chapter should have some (probably not all) contributions discussed.  Be obvious; use headings.

Knowledge - background:

• justify choices
• explain methods
• acknowledge alternatives
• evaluate

Evidence, well-reasoned arguments, acknowledge limitations.

Clear openings for future work.  Be clear where they are.

Make it reproduceable.

Short / concise.  Examiners like short theses.

Introduce what's interesting and important.

When outline thesis, look at structure of main argument, not of document.

Background material must have point.  Only include as much detail as you need to make point.
Points, eg:

• Explain method you use.
• Novelty of your approach. Similarities with existing work.
• Justify choices (evaluate other work).
• Don't tear down others' work. 'Build on'.
• Cite examiners, they've probably published something relevant.. (but not for the sake of it).

Then we had five minutes to write down what our PhDs are about and what we have already found out.  I wrote:

How do the futures of the Semantic Web and amateur digital content creation fit together?
Can Semantic Web tools and technologies be used to enhance collaborative creative partnerships and encourage fruitful outputs?

There are knowledge sharing systems and collaborative tools for scientific fields and in education, but nothing for creative artsy things.

Attitudes towards data sharing and privacy amongst content creators are in flux.  There are lots of projects and energy around open data and decentralised social networks that allow data to become portable and not tied to one platform.  One of TBL's visions for the Semantic Web is the dissolution of data silos and 'walled' applications that disadvantage the user, and as such the promotion of the 'ownership' of a user's data by the user themselves, rather than the software or organisation that uses the data.

There are lots of reasons people make content.  There are lots of reasons people don't make content (who could / would like to).

[Notes resume]
Use backreferences; don't repeat yourself.

Info / advice
...homepages.../sgwater/resources.html
..homepages.../imurray2/teaching/writing
Style: Toward Clarity & Grace (book)
The Craft of Research (book)

When to start writing thesis?

• Do you already have papers?  Slot them into a thesis template asap.
• Maybe a year beforehand.  Slower pace is better.

Don't assume appendices will be read.  More for extra info if needed by people trying to reproduce your work (not your examiners).

Too many direct quotes look like you don't understand and are avoiding explaining yourself.

Keep copies of web resources and cite access dates in case they change / disappear.
Figures might be copyright if you just copy them from papers, even if you cite them.  Remake them, and put 'adapted from' as citation.

Examiners?

• Depends on your supervisor.  Discuss.  Student might be able to suggest someone to examine.
• Maybe a balance between internal and external knowledge.
• Won't be someone junior, even if they're considered an expert in the field.
• Helpful if supervisor knows how that person will behave in viva.  Might be a good reason to avoid someone you think would be perfect from their background.
• Conflict of interest regulations.  You can know them personally though.  External can't have been affiliated with UoE in the last three years, or substantially involved in your research (like co-authoring a paper).  No ex-supervisors, from any university.

No grading system (ie no different levels of passed PhD).  Might be external prizes if you want extra recognition.

25th Feb - 3rd March

I spent most of this week in a remote village by the sea in the Scottish Highlands.

When I came back I read and made notes about the Semantic Web and social machines by TBL; and typed up some notes from a while ago about OntoMedia and the Semantic Web and communities by K. Faith Lawrence.

I also translated all of my written meeting notes into Evernote, which promptly glitched out and doubled the amount of typing I had to do.  (I considered switching back to Google Docs, but I need labels).  I sure love technology.

I refamiliarised myself with the structure OWL.  Awesome diagrams here.

I did some more thinking about how I need to work with amateur content creators to make an ontology that fits their workflow.  I should have finished the planning stage of this ages ago, but.. I blame the ILWhack.

I keep wondering about the best way to have a system of consistent URIs across a network where the information can move from server to server on the whim of a user.  During this wonderment I discovered that purl.org's login system is broken.  I joined the mailing list, and people complain about it and have it re-fixed fairly regularly, so I'll just wait..