We were introduced to the various mini-projects on Tuesday morning, and encouraged to form teams with people who weren't from the same university. I quickly shortlisted the five that sounded most interesting to me, but was disappointed that there weren't any about multimedia. Because how to evaluate a very subjective system is a potential problem for me, the project proposed by Valentina Presutti was my first choice:
"Serendipity can be defined as the combination of relevance and unexpectedness: an information is considered serendipitous if it is at the same time very relevant and unexpected for a given user and in the context of a given task. In other words, a user would learn new relevant knowledge. To evaluate the performance of a tool (e.g., an exploratory search tool, a recommending system) in terms of its ability to provide users with serendipitous knowledge is a hard task because both relevance and unexpectedness are highly subjective. This miniproject focuses on two main research questions: what is the correct way of designing a user-study for evaluating an exploratory search tool performance in terms of serendipity? Is it possible to build a reusable set of resources (a benchmark) for evaluating ability to produce serendipity, allowing easier evaluation experiments and comparison among different tools?"
Nobody else seemed to be interested though, so I resigned myself to not being able to do it... until I explained the project and why it was interesting, to the best of my ability, to Andy, Oscar and Josef, and they were sold enough to mark it as our first choice. Thus Team Anaconda Disappointed (a name of significant and mysterious origins) was born, and Project Cusack (because of the movie Serendipity, which nobody got) was underway.
Our first lecture today was from Lynda Hardman, about telling stories with multimedia objects. It was super relevant to what I'm doing, to the point where I'm surprised I hadn't come across her work already. My notes are here. Lynda has done, for example, work with annotation of personal media objects like holiday photos in order to combine them into a media presentation. She has considered similar things to me, in particular noting that there are many many aspects of data about multimedia - I had assembled my take on this into a Venn diagram for my poster..
One I hadn't considered is annotating an explict message of a piece of media, intended by the creator. This isn't always relevant - sometimes the consumer's interpretation of the media is more important - and this in itself might be an interesting annotation problem. Competing perspectives - something an ontology should be able to represent.
I need to check out COMM - Core Ontology for Multimedia.
She has an overview of the canonical processes they have consolidated the process of producing digital content into, and how annotation can be formed around these.
Lynda also told us about Vox Populi and and LinkedTV; practical applications of annotating multimedia.
I made lots and lots of notes.
Natasha Hoy gave us some insights from the biomedical world with regards to ontology development, particularly in relation to the International Classification of Diseases which, when last revised in the 80s, consisted of a lot of paper and a whoever-shouts-the-loudest algorithm for inclusion of terms. But the next version, currently under creation, is being developed with a version of Web Protege, customised to be friendly for those who don't know or care about ontologies, and is a truly collaborative process (for those allowed to take part) with accountability for all changes. It's open too though, so even those without modification rights can view and comment on the developments. My notes are here.
Lunch was for the first time outside, under the shadows of the forest, and for me was a tray of tomatoey vegetables that were delicious but few. A striking contrast to Monday's lunch. Everyone else had some meat-potato combination, preceded by a salad with tuna, and followed by a peach.
The hands-on session followed on from Natasha's talk. We teamed up (temporarily Anaconda Hopeful) and played with Web Protégé. There were two magazines and two newspapers, each with four departments. Anaconda Hopeful were randomly designated the Advertising Department of Iberia Travel (a food and travel magazine). We got stuck in, on paper first to identify some classes and relations that were relevant to us, and then with Web Protégé, along with the other departments of Iberia Travel. We didn't come into any conflicts, but ended up creating a few classes that we needed, but should really have been the remit of another department (I guess we just got there first).
Then it was announced that Iberia Travel had bought the other magazine (and one of the newspapers had bought the other), and we had to work together to merge ontologies with the other department. It became apparent that the other magazine had never had an Advertising Department (no wonder they went under!) so we had no-one to attempt to merge ontologies with. We attempted to sell our expertise to the Advertising Departments of the newspapers, but there were already too many people involved in the heated debate that came out of the ontology merging there, so we couldn't really get involved.
Later we got cracking with our mini-projects. Valentina showed us aemoo, and the experiments her team had come up with to try to evaluate it. We sat down by ourselves to brainstorm, describing a lot of concepts for ourselves, breaking down the notion of serendipity, figuring out what might be wrong with existing experiments to 'measure' serendipity, and collating literature in the area. (Turns out there is a lot, and it's a very interdisciplinary issue; lots to read about from social sciences, anthropology etc, as well as philosophy of science. In computing, it seems to be primarily discussed within the realms of recommender systems and exploratory search).
Serendipity seems to be mainly described as a combination of unexpectedness and relevance. Problems include the sheer subjectivity of it. Some people are going to get excited by all facts they find out, whether they're useful or not. Some people are going to have hidden, inexplicit or subconscious goals that affect how 'relevant' something is to them. People describe their different areas of expertise in different ways; some are more humble than others and would not call themselves an expert in a topic, for example. So whether or not an event can be considered a serendipitous one is a complex question, which must take into account the person's background, goals and existing knowledge, the task they are trying to achieve (or lack thereof, as serendipity is particularly important - in my opinion - in undirected, loosely-motivated activities), the way they are able or encouraged to interact with a system, what they are doing before and after... all these things make up a context for someone's activities, and none of them seem to be particularly measurable.
Dinner was a vegetable and potato (yay!) starter, followed by spaghetti in tomato sauce (fish for everyone else, although Andy got a custom omelette, lah-de-dah). Also an apple. We learnt the hard way not to sit at a table directly underneath a light, as the bugs just raiiiin down.
After dinner we crowded around Enrico who had offered to provide advice about PhD-ing. From this session, I have a signed diagram of the life of a PhD, because he borrowed my notebook to make it. I tuned in and out of the discussion, and noticed some irregularities between my PhD and what seemed to be 'normal'. For instance, most people didn't seem to have as much control over their topic, or what they were doing at any given moment in their first year. I am really, really enjoying my freedom, but in order to justify that I deserve it I need to sort out my lack of direction and focus. I need to believe in what I'm doing - not be told by someone else - which is one of the main reasons I am doing this particular PhD. Perhaps I need to ask for more guidance to more quickly reach the necessary conclusions for myself. (And, of course, perhaps I also need to stop taking big chunks of time out periodically for different reasons; that might speed up the process as well).
Later, the overriding sentiment was that the job of a PhD student was to answer a question, to produce a theory. Not to create a system or solve a large problem; certainly not to worry about practical, real-world applications of theories. Well, I've already explained that this is something I can't accept, and I still am not convinced that that is going to impact on my ability to do a PhD. Theories develop during practice. Coding and designing, like writing, are part of my thought processes, and I reach realisations or find new questions to ask through hacking and playing and making. And why would I be hacking and playing and making, if not to try to produce something of real-world value? If my motivation in making a system is explicitly to come up with new theories, then my approach and outcomes and realisations will be entirely different. In trying to make something that works for real people, not researchers in a restricted domain or specific context; a clean and sterile laboratory, I figure out different things, that matter.
There was another discussion that I came into a bit late, but it sounded like a very harsh discussion about problems with research in industry (rather than academia) that seemed to be very overstated compared to what I have read and experienced myself.
By the end of the day, it felt like I'd been at Summer School for weeks, and had known everyone forever.
"Serendipity can be defined as the combination of relevance and unexpectedness: an information is considered serendipitous if it is at the same time very relevant and unexpected for a given user and in the context of a given task. In other words, a user would learn new relevant knowledge. To evaluate the performance of a tool (e.g., an exploratory search tool, a recommending system) in terms of its ability to provide users with serendipitous knowledge is a hard task because both relevance and unexpectedness are highly subjective. This miniproject focuses on two main research questions: what is the correct way of designing a user-study for evaluating an exploratory search tool performance in terms of serendipity? Is it possible to build a reusable set of resources (a benchmark) for evaluating ability to produce serendipity, allowing easier evaluation experiments and comparison among different tools?"
Nobody else seemed to be interested though, so I resigned myself to not being able to do it... until I explained the project and why it was interesting, to the best of my ability, to Andy, Oscar and Josef, and they were sold enough to mark it as our first choice. Thus Team Anaconda Disappointed (a name of significant and mysterious origins) was born, and Project Cusack (because of the movie Serendipity, which nobody got) was underway.
Our first lecture today was from Lynda Hardman, about telling stories with multimedia objects. It was super relevant to what I'm doing, to the point where I'm surprised I hadn't come across her work already. My notes are here. Lynda has done, for example, work with annotation of personal media objects like holiday photos in order to combine them into a media presentation. She has considered similar things to me, in particular noting that there are many many aspects of data about multimedia - I had assembled my take on this into a Venn diagram for my poster..
One I hadn't considered is annotating an explict message of a piece of media, intended by the creator. This isn't always relevant - sometimes the consumer's interpretation of the media is more important - and this in itself might be an interesting annotation problem. Competing perspectives - something an ontology should be able to represent.
I need to check out COMM - Core Ontology for Multimedia.
She has an overview of the canonical processes they have consolidated the process of producing digital content into, and how annotation can be formed around these.
Lynda also told us about Vox Populi and and LinkedTV; practical applications of annotating multimedia.
I made lots and lots of notes.
Natasha Hoy gave us some insights from the biomedical world with regards to ontology development, particularly in relation to the International Classification of Diseases which, when last revised in the 80s, consisted of a lot of paper and a whoever-shouts-the-loudest algorithm for inclusion of terms. But the next version, currently under creation, is being developed with a version of Web Protege, customised to be friendly for those who don't know or care about ontologies, and is a truly collaborative process (for those allowed to take part) with accountability for all changes. It's open too though, so even those without modification rights can view and comment on the developments. My notes are here.
Lunch was for the first time outside, under the shadows of the forest, and for me was a tray of tomatoey vegetables that were delicious but few. A striking contrast to Monday's lunch. Everyone else had some meat-potato combination, preceded by a salad with tuna, and followed by a peach.
The hands-on session followed on from Natasha's talk. We teamed up (temporarily Anaconda Hopeful) and played with Web Protégé. There were two magazines and two newspapers, each with four departments. Anaconda Hopeful were randomly designated the Advertising Department of Iberia Travel (a food and travel magazine). We got stuck in, on paper first to identify some classes and relations that were relevant to us, and then with Web Protégé, along with the other departments of Iberia Travel. We didn't come into any conflicts, but ended up creating a few classes that we needed, but should really have been the remit of another department (I guess we just got there first).
Then it was announced that Iberia Travel had bought the other magazine (and one of the newspapers had bought the other), and we had to work together to merge ontologies with the other department. It became apparent that the other magazine had never had an Advertising Department (no wonder they went under!) so we had no-one to attempt to merge ontologies with. We attempted to sell our expertise to the Advertising Departments of the newspapers, but there were already too many people involved in the heated debate that came out of the ontology merging there, so we couldn't really get involved.
Later we got cracking with our mini-projects. Valentina showed us aemoo, and the experiments her team had come up with to try to evaluate it. We sat down by ourselves to brainstorm, describing a lot of concepts for ourselves, breaking down the notion of serendipity, figuring out what might be wrong with existing experiments to 'measure' serendipity, and collating literature in the area. (Turns out there is a lot, and it's a very interdisciplinary issue; lots to read about from social sciences, anthropology etc, as well as philosophy of science. In computing, it seems to be primarily discussed within the realms of recommender systems and exploratory search).
Serendipity seems to be mainly described as a combination of unexpectedness and relevance. Problems include the sheer subjectivity of it. Some people are going to get excited by all facts they find out, whether they're useful or not. Some people are going to have hidden, inexplicit or subconscious goals that affect how 'relevant' something is to them. People describe their different areas of expertise in different ways; some are more humble than others and would not call themselves an expert in a topic, for example. So whether or not an event can be considered a serendipitous one is a complex question, which must take into account the person's background, goals and existing knowledge, the task they are trying to achieve (or lack thereof, as serendipity is particularly important - in my opinion - in undirected, loosely-motivated activities), the way they are able or encouraged to interact with a system, what they are doing before and after... all these things make up a context for someone's activities, and none of them seem to be particularly measurable.
Dinner was a vegetable and potato (yay!) starter, followed by spaghetti in tomato sauce (fish for everyone else, although Andy got a custom omelette, lah-de-dah). Also an apple. We learnt the hard way not to sit at a table directly underneath a light, as the bugs just raiiiin down.
After dinner we crowded around Enrico who had offered to provide advice about PhD-ing. From this session, I have a signed diagram of the life of a PhD, because he borrowed my notebook to make it. I tuned in and out of the discussion, and noticed some irregularities between my PhD and what seemed to be 'normal'. For instance, most people didn't seem to have as much control over their topic, or what they were doing at any given moment in their first year. I am really, really enjoying my freedom, but in order to justify that I deserve it I need to sort out my lack of direction and focus. I need to believe in what I'm doing - not be told by someone else - which is one of the main reasons I am doing this particular PhD. Perhaps I need to ask for more guidance to more quickly reach the necessary conclusions for myself. (And, of course, perhaps I also need to stop taking big chunks of time out periodically for different reasons; that might speed up the process as well).
Later, the overriding sentiment was that the job of a PhD student was to answer a question, to produce a theory. Not to create a system or solve a large problem; certainly not to worry about practical, real-world applications of theories. Well, I've already explained that this is something I can't accept, and I still am not convinced that that is going to impact on my ability to do a PhD. Theories develop during practice. Coding and designing, like writing, are part of my thought processes, and I reach realisations or find new questions to ask through hacking and playing and making. And why would I be hacking and playing and making, if not to try to produce something of real-world value? If my motivation in making a system is explicitly to come up with new theories, then my approach and outcomes and realisations will be entirely different. In trying to make something that works for real people, not researchers in a restricted domain or specific context; a clean and sterile laboratory, I figure out different things, that matter.
There was another discussion that I came into a bit late, but it sounded like a very harsh discussion about problems with research in industry (rather than academia) that seemed to be very overstated compared to what I have read and experienced myself.
By the end of the day, it felt like I'd been at Summer School for weeks, and had known everyone forever.
Post a Comment