I find the M3 approach very appealing and practical, it seems very systematic and probably most useful when the main objectives for an intervention can be reasonably well-specified fairly early on in the project?

I had some questions about tensions between the levels and I guess these are really to do with how the evaluation framework feeds into design. You give nice examples showing how some issues identified through the M3 approach may be resolved by making changes at different levels (e.g. changing the functionality/usability of the design, changing the activity/learning design, or possibly by making changes to the organisational setting). I guess which level/combination of levels you choose to address any issue at depends very much on objectives and practicalities of the situation. However, I wondered whether you have guidance about which levels should be prioritised? For example, should we usually aim to prioritise the meso level of learning experience - get this right first (or through an iterative process) and aim to make adjustments at the micro (usability) and macro (organisational) level so as to make the 'ideal' meso level experience work in practice? Obviously, in reality there are interdependencies here as in we may not be technically capable of delivering the ideal design (at the micro level) or capable of making/forcing changes at the macro level - and I think your framework recognises these interdependencies. On a slightly related note, I wonder whether sometimes are 'ideal' designs our not visible enough in the literature (e.g. this is what we really wanted to do (and evaluate) but this is what we did because of these technical/practical/organisational constraints?

Finally, you describe the micro level as being concerned with usability - do you distinguish at this level of evaluation between functionality (whether it is possible to do something with the technology) and usability (ease-of-use/whether the functionality is used)?

Joshua Underwood 432 days ago