Showing posts with label research methods. Show all posts
Showing posts with label research methods. Show all posts

Tuesday, 7 July 2015

Project preparation and how to read a paper

Followers will know I think reading and writing are rather important.

Project students at all levels, postgrads and interns all need to work on this, not to mention post docs and academics.

Here are some pointers to materials which can be used as foundational tools for those wishing to update their skills, or just think and reflect.

Practically the FutureLearn MOOC on Developing Your Research Project is helpful. Although, I do take exception to the supporting material which talks about learning styles, but that is the theme for another post.

https://www.futurelearn.com/courses/research-project

A team of academics developed an online course on research methods which is well worth a skim read with some drilling down to explore issues (just like you might use a text book).

http://www.erm.ecs.soton.ac.uk

Moving on to the practicalities of reading academic papers, I can point to two useful resources,

  1. A short practical piece by Michael Mitzenmacher of Harvard

http://www.eecs.harvard.edu/~michaelm/postscripts/ReadPaper.pdf

 

2. A paper presented at a CS conference titled "How to read a paper" by Srinivasan Kerchav from Waterloo in Canada.

http://ccr.sigcomm.org/online/files/p83-keshavA.pdf

 

Monday, 16 January 2012

correlation

One of the modules I teach is research methods to our Web Science Masters - we cover qualitative and quantitative methods

I like this

 

correlation.png

http://imgs.xkcd.com/comics/correlation.png

correlation.png

Monday, 21 March 2011

Time for some publications

Bit of a round up on advice on where to publish and how to publish, seems timely revision, we are never too old to learn!
Also thought this might be useful to friends, colleagues and of course PhD students.
I have a couple of papers in the mix at the moment, and am on a mission to get some decent publications out there while I am on sabbatical at LIRMM in Montpellier, so its in focus for me just now.
Seems that Abby Day (formerly of Lancaster, and now based at the University of Sussex) got herself a bit of a reputation a few years ago advising people who to get published in Journals.  You can get a short version of her advice in a downloadable article titled How to write publishable papers you can see a snapshot below.
Actually a bit more searching located all three parts of her advice via the author pages of  The International Education Journal: Comparative Perspectives (IEJ).
Screen shot 2011 03 22 at 22 18 14
Day points out that some of the empirical basis for the article's guidance are derived from research which was sponsored by Emerald which looked at the quality indicators of academic journals Day, A. and Peters, J. 1995 Quality indicators in academic publishing', Library Review, vol 45 no 3/4.
Emerald, like many journal publishers has author pages which include a set of writings on the topic of how to get published and disseminate your work.

My post was actually prompted by the fact that pubications are on the top of my todo list just now, and that I came across a (timely) tweet pointing me to a Prezi which claimed to be able to help "
Screen shot 2011 03 21 at 16 10 01

The source of this work is a researcher from Melbourne who tweets as the @thesiswhisperer (great handle isn't it) and who had a blog of the same name and is known in the real world as Inger Mewburn (again a distinctive handle, but perhaps less easily recalled).
The prezi has
  • a few links to related work - a post on Publish2PhD
  • anatomy of types of papers
  • reference to helping Doctoral Students to Write, Kamler and Thompson (writing a tiny text)
  • writing a spew draft - links to a post about using scrivener to help in this process ( and had me resolving to take another look at tinderbox)
  • writing a scratch outline
  • cleaning the mess - clarify your ideas - may be itterative
  • murder your darlings (edit and revise)
  • leave it to relax/proove/rest (we are talking bread making analogies here) - critical friends too
  • this prezi is not too much like 'powerpoint on acid' - others can be. I find myself thinking how long did it take her to create that.....
I think this post might usefully be linked to something I identified last year which was related to a learning technologies roadmap, and also various pieces on places to publish. More of that later...

Refs:
Day, A., 1996, How to get research published in Journals, Gower, Aldershot, UK
Day, A. and Peters, J., 1995 Quality indicators in academic publishing', Library Review, vol 45 no 3/4
Brown S., Black., Day A., Race P., 1998, 500 Tips for getting published: A guide for educators, researchers and Professionals, Kogan Page, London, UK


Wednesday, 9 June 2010

readings: Modified Delphi Technique

Delphi technique goes back to the Rand Corporation in the US just post 1939-1945 war, although I guess the reference to the original delpi (the ancient greek oracle) is indicative of the roots of this approach.

There is an intersting paper which exemplifies an early(ish) application of the modified delphi in the context of higher education.

Alfred R Hecht, A Modified Delphi Technique for Obtaining Consensus on Institutional Research Priorities. Research Brief. Paper presented at the Annual Meeting of the North Central Region AERA Special Interest Group on Community College Research, July, 1977
http://eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED140912

The paper is rather old, but reads quite well and is clear. Additionally its quite instructive to read a 1977 paper and see what it looks like! personally, I love the way that people like ERIC and ACM Digital Library are making digitisations of older papers available as PDF images.

As a more recent  (tho' still fairly old) paper which is relevant to modern day applications
Custer et all 1999 is useful JVTE v15n2: The Modified Delphi Technique - A Rotational Modification

The wikipedia ref is OK but probably not helpful to the typical undergrad working on a project.

Linstone and Turrof’s book is online (for which some of the chapters may be useful, but which is probably a bit too much for you to read http://www.is.njit.edu/pubs/delphibook/

Why use the modified Delphi?
  • Basically focus groups can be problematic
  • Members can skew the responses by influencing group dynamics
Delphi and modified delphi is also sometimes knows as quick consultancy

A useful brief account of the technique can be found in A Handbook of Techniques for Formative Evaluation By Judith W. George, John Cowan
  • It is possible to run it in either entirely remotely with deadlines on the voting and responses
  • Or to run it partly electronically and partly face to face, where you do the setup electronically, but then go through the consultation in a group in a fairly short time. 
  • Its a very efficient approach to get a good volume of evaluation data
Also worth considering - Nominal Group Technique - see wikipedia entry as a starting point

I plan to put some more info on my blog elaborating ways to use the approach, which will include a growing set of refs. You may also want to look at the TELUSS project

I would be interested in a student project to develop a tool to manage all the stages of the process of a modified delphi technique

Tuesday, 8 June 2010

Visual Thinking Revisited



This is an update on my previous blog on visual literacy. For me, I think that high quality communication is key in every aspect of life, personal and professional, this post includes some thoughts, some references and some examples.

Why? 1) I'm interested in learning, and technology and change; 2) I'm active in teaching and researching learning technology and change

that means that I spend quite a lot of time thinking about what is happening, and trying to make sense of the evidence I find,
I have a set of visualisation tools I regularly use and thought it would be good to learn their proper names and actually categorise them

I was prompted to do this by a variety of experiences, but primarily one excellent resource (the periodic table of visualisation methods) and numerous bad experiences of me, my colleagues and students presenting information in unintelligible ways

Making Sense - Visualisation as a tool for thinking

cut a bit of slack here, using the wrong tool is sometimes ok, because its about a work in progress, making sense of your information and trying to find ways of understanding and communicating what you have found. After all playing with different sorts of visualisations will give you personal experience on which you can refine your understanding.

Communicating Understanding - Visualisation as a tool for talking

when we want to enter into a discourse about our understanding, then finding the best way to present information to afford that discourse is quite useful

while I am doing that, I would like to quote Tufte's observation "Graphical excellence is that which gives to the viewer the greatest number of ideas in the shortest time with the least ink in the smallest space"

So....

Data Visualisation

Expressing ideas in terms of visualisation can be a powerful tool for deepening your own understanding, and also for communicating new ideas.
Visualisations can range from the strictly factual (e.g. a graph derived from a set of data) through to ideographic (explaining the perceived inter-relationship between sets of ideas and concepts)
In some ways a table which summarises analysis of artefacts (a set of reviewed software or a set of reviewed papers) can be seen as a type of visualisation.

Visualisations can help save words in written reports, and can also be used to help structure an argument in a paper or a visual presentation. Most importantly they can contribute effectively to the communication of ideas, stimulating debate and disseminating understanding.

References

A few references for visualisations are listed below:


Tools List

Periodic Table of Methods of Visualisation categorises visualisations into six broad types.
  • Data
  • Information
  • Concept
  • Strategy
  • Metaphor
  • Compound
This post looks at visualisation methods which I have found useful and relevant to my academic activities, either for research or for marshalling arguments and explanations either for teaching or explaining my understandings to friends and colleagues. It is interesting to consider this collection of methods against the repertoire of methods which are routinely used in particular disciplines (e.g. Computer Science and Management) In some cases, where such methods have been specifically designed to communicate a formal development process, or to document and subsequently manage a development process there is a stronger degree of literality and rigour, than might be found in some of the conceptual methods/thinking tools such as mind maps.
  • One interesting role which has emerged out of the field of graphic visualisations is that of the graphic facilitator.
  • Methods such as graphics cafes are also interesting.
Data Visualisation

These visualisations provide a direct mapping between the information which is presented, and the data which was collected and analysed.

Basic visualisations which most students  produce include tables, graphs, histograms, pie charts, and magic quadrants.  They are probably the most common visualisations which you will find in published papers. Often we are as more constrained by our medium of presentation (eg double column journal pages) that we are by the objective of clear communication.


Tables, Pie Charts and stuff

This is data representation, there is a one to one relationship between the data and the representation. With tables we can choose information, and communicate our undertanding by the location of information, we may also gain by putting a lot of data in one place, which people can use as a signpost, when they then follow an explanation through the text.

Concept Visualisations
I am particularly interested in concept visualisations because I am intersted in the communication of ideas - when we speculate and theories we are dealing in the conceptual. Some visualisation methods are more rigorous than others being associated with specific research practices' accepted analytical methods.  Others function as thinking tools, which represent incomplete (or not yet complete) representations of understandings.  

Concept Map
This is a formal modelling tool commonly used for knowledge representation and ontology creation.

Mind Map

MikeEllisMindMap110057518.jpg This MindMap from Flickr courtesy of Mike Ellis from http://electronicmuseum.org.uk . The mindmap approach was designed originally and championed by Tony Buzan. Mind mapping is a thinking tool which can also be used for formally to record information spaces and to manage workflows. Its worth looking at the Buzan web site as a follow up, mindmaps may be drawn by hand or using software, and there are many tools available which can be used to create mind maps on computers. The diagram linked from in the Radar Diagram (below) provides an interactive snapshop of mapping software 

you probably already know about the following, but do some web searching if you are in doubt!
  • Venn Diagram
  • Cluster Diagram
  • Layer Chart
  • Concentric Circles

Radar


Picture 13.png


from
http://www.visual-literacy.org/pages/maps/mapping_tools_radar/radar.html

Compound Visualisations:

According to the visualisation periodic table, there are six types of compound visualisations - although I think that if you go to Tufte he identifies quite a few which have been generated with firm mappings to their data source.
 

Knowledge Map
These are basically hypothetical maps (often in the style of maps created by early mariners) which seek to demonstrate the 'landscape'. They also remind me in style of maps which accompany books like The Lord of the Rings and Swallows and Amazons.


You probably need a great deal of imagination to create a convincing knowledge map, but they can be highly persuasive and powerful in communicating an overview of content and issues in a particular area.


I find label knowledge maps a little misleading, and wonder if mythical maps, or metaphor maps might be more accurate. To me the term knowledge implies a degree of certainty and finality which I do not think is actually communicated in the final product.


I have come across a couple of knowledge maps which are probably of interest to folks in my research area.


Knowledge Map



webmap.png



Permanent link to this comic: http://xkcd.com/256/

Image URL (for hotlinking/embedding): http://imgs.xkcd.com/comics/online_communities_small.png

eLearnland



elearnland.png



Learning Map



early work in this area was done by companies seeking to explain the intricacies of their organisaiton. Probably the most famous is the work done by Pepsi on Beverage Street. There is a paper which explores this work The Learning Map Approach by James Haudan and Christy Contardi Stone, a white paper published in The Change Handbook, Peggy Holman, Tom Devane and Stevan Caddy. Here the approach is one of metaphor rather than formal modelling, although it may be possible to incorporate meaning via metaphor, such as sense of proximity and distance, known and unknown.



LearningMapinformal-learning.jpg



from
http://elearningargentina.files.wordpress.com/2008/03/informal-learning.jpg

Underground Map



This is called a metro map by the folks at visual-lliteracy.org, but since the original map (a visualisation which depicts conections and interconnections, but priviledges this over scale accuracy) was designed for the London Underground, I think it is more accurate and respectful to call it an Underground Map - the paris metro map is a totally different beast!

This example visualisation of Web Trends was produced in 2007 by Otto Nassar (version 2)

WebTrendsUndergroundMap.png


According to the web site at http://informationarchitects.jp/wtm4/
"The map has been featured all across the web from .... The Web Trend Map plots the Internet’s leading names and domains onto the Tokyo Metro map. Domains and personalities are carefully selected through dialogue with map enthusiasts, and every domain is evaluated based on traffic, revenue, and character. "

Version 4 - a rather different beast is shown below



Other Stuff

Information is Beautiful

Picture 26.png

rather fabulous site to browse to gather ideas and models of information graphics

Colorschemer












if you are going to present visual information, you will need to choose decent colour schemes! take a look at http://www.colorschemer.com/blog/

visualising words and their semantic interconnections

tag cloud
just as you see on the left of this post, it offers both a mapping and a compact visual analysis of the information



"Wordle is a toy for generating “word clouds” from text that you provide. The clouds give greater prominence to words that appear more frequently in the source text. You can tweak your clouds with different fonts, layouts, and color schemes. The images you create with Wordle are yours to use however you like. You can print them out, or save them to the Wordle gallery to share with your friends".




Gallery of Data Visualization - the best and worst of statistical graphics

Gapminder is an online visualisation tool. It works with a collection of provided data sets, useful for getting an idea of what visualisations might look like.

visualcomplexity.com is a website which provides an index into many different visualisation methods and tools


The last time I looked, the main categories of visualisation were
Art (62)
 Biology (50)
 Business Networks (25)
 Computer Systems (29)
 Food Webs (7)
 Internet (30)
 Knowledge Networks (105)
 Multi-Domain Representation (60)
 Music (33)
 Others (59)
 Pattern Recognition (24)
 Political Networks (20)
 Semantic Networks (30)
 Social Networks (89)
 Transportation Networks (45)
 World Wide Web (55)




Further reading 
 Concept Mapping with thanks to the online concept mapping resource guide http://www.socialresearchmethods.net/mapping/mapping.htm

-->

Trochim, W. Reliability of Concept Mapping. Paper presented at the Annual Conference of the American Evaluation Association, Dallas, Texas, November, 1993. 

Tukey, John W (1977). Exploratory Data Analysis. Addison-Wesley. ISBN 0-201-07616-0. OCLC 3058187.

Tuesday, 16 December 2008

TEL Tales: stronger methods for understanding 2.0 Social Networks

not sure about this, but think I will be posting my TEL stuff on tell tales, and my running stuff on runningsu
wonder then what I should put on shirley knot...
just for today I think I will put tell tales here too, then maybe someone with wisdom could advise me on the best thing to do....

Towards stronger methods for understanding Social Networks - today and tomorrow

Been talking with one of my students about doing some facebook analytics as a part of a research project looking at social networking. One of the things which interests me is the appropriate research methodologies to actually get to the bottom of why and how people use these sorts of networks.

Two publications in the near past (Dec 2009) give clues to why we need to get our research methods sorted.
1) Peter Schwartz in the Huffington Post blogs on the proposed death of facebook (well I paraphrase)

2) a paper from Huberman et al on Social Networks that matter: Twitter under the microscope

so my student wants to do a bit of traversing of networks and mapping patterns of connections. and it would be good if we could identify any markers for behaviours - there have been a few posts on that stuff too - something like "ten types of twitter users" I seem to recall

And after we have collected our first set of data from the network traversal, we want to talk to different user types and get some backfill on their actual usage, motivations etc etc. The whole task would be monumental - or at least suitable for some extended study, but this is going to be short and to the point.

The issue I have with both the Schwartz blog, and Huberman et al, it that they leave dangling more questions that they actually answer.

Looking at what Schwartz offers in anecdotal evidence about user preferences, what Huberman et al suggest about evidence of real network (the little networks within networks that really matter - my phrase not theirs) reminds me that I also have anecdotal and personal ethnographic evidence of a more complex set of connections, and values.

and what I want is a method of exploring this sort of network, both by physically getting the evidence from the networks, and from identifying and gathering personal data from Representative selections of those users.

I 'know' from discussions with active users whose posts I value, that there is mileage in this area, the question for me (and my student) is how are we going to get to that precious data, and therefore, what can we do to get a bit further down the methodological minefield