Open Questions and presentation guidelines

Some open questions:

• Web 2.0 'mass cooperation' is different from small group CSCL: how does that affect the indicators?

• Where do "norms' for feedback indicators come from? How can participants affect the norms? (In general, this is also important to answer for indicators for researchers.)

• The trap of KPIs: How to avoid "playing the (numbers) game"?

• Temporality and sequentiality: How to deal with time?

• Reflexivity: What is it participants "react" to?

• IT: Indicators and visualisations should become web services: how to get there?

Questions to the (tool) presenters:

• For which questions do your indicators/visualisations provide answers? What function(s) do they fulfill (best)?

• In case the indicators are grounded in a (learning/collaboration) theory: Which one is it?

• Are the visualisations available for "off-line" use or can they be used for continuous monitoring?

• What are the requirements on the data used to compute the visualisations?

• What are the requirements on the tool itself (such as operating system)?

• What kind of research is needed to help you developing the approach further? Short term and Long term.

Questions that the "viewers" of tool demos should answer

(Best on a rating scale printed on paper)

• Ease of use (very easy....very hard)

• Learnability/amount of training required (little instruction required ... intensive instruction required)

• Visualisations appropriate to be used as feedback to students? (easy to understand by students .... hard to understand by students)

• How much 'inference' goes into the indicators? (Face valid ...... highly inferential)

• From the indicators provided, these are the most relevant indicators based on my experience:...

• The tool with its indicators and visualisations seems particularly appropriate for these kind of (learning and/or research) settings:....