Educators Embrace Multiscale Analytics to Help Students Understand Their Design Processes Better

Table of Links

Abstract and 1. Introduction

1. Prior Work and 2.1 Educational Objectives of Learning Activities

   2.2 Multiscale Design

   2.3 Assessing Creative Visual Design

   2.4 Learning Analytics and Dashboards

2. Research Artifact/Probe

   3.1 Multiscale Design Environment

   3.2 Integrating a Design Analytics Dashboard with the Multiscale Design Environment

3. Methodology and Context

   4.1 Course Contexts

   4.2 Instructor interviews

4. Findings

   5.1 Gaining Insights and Informing Pedagogical Action

   5.2 Support for Exploration, Understanding, and Validation of Analytics

   5.3 Using Analytics for Assessment and Feedback

   5.4 Analytics as a Potential Source of Self-Reflection for Students

5. Discussion + Implications: Contextualizing: Analytics to Support Design Education

   6.1 Indexicality: Demonstrating Design Analytics by Linking to Instances

   6.2 Supporting Assessment and Feedback in Design Courses through Multiscale Design Analytics

   6.3 Limitations of Multiscale Design Analytics

6. Conclusion and References

A. Interview Questions

\

5.3 Using Analytics for Assessment and Feedback

The more design classroom sizes continue to grow, the more that instructors are challenged in having time to provide optimal levels of feedback to each student [47]. Prior studies have found analytics useful toward scaling assessment and feedback [49, 60]. Our findings for multiscale design analytics align. I1 and I4 expressed that the analytics can become a part of their rubrics and feedback they give to students. Further, according to I4, making these analytics a part of rubrics can motivate and provide students guidance on what instructors are looking for in their design.

\ I1: I think [these analytics and my rubrics] complement each other. I think it will be very helpful…if there’s a way that I can just sort of make a rubric on [dashboard] and attach to when they get their feedback.

\ I4: You know, give them something to shoot for…I think that I would say…here are the things that I’d like to see in your design…I think that I would definitely like to assign scales as a part of the rubric to say, I would like to see the big picture from out here, and then when you zoom in, see more.

\ We observe the potential of multiscale design analytics toward expediting instructors’ assessment work. I9 preferred to utilize analytics as a quick-to-use indicator of underlying problems. The analytics help them reduce the time they otherwise would spend on assessing each design.

\ I9: So, I won’t use the values in the column to directly give them points…But it’s better than having to go to every [design] and look for every single issue or having a much larger rubric that I ran by…So think of the analytics as the symptoms and [then] you actually identify diseases.

5.4 Analytics as a Potential Source of Self-Reflection for Students

Instructors (I1, I4, I5, I6) in our study expressed expectations that students would benefit from seeing multiscale design analytics. According to them, seeing analytics has the potential to help students reflect on their progress. More specifically, seeing analytics can help students in becoming aware of how they are organizing their ideas spatially across scales and clusters. Self-reflection through analytics plays a vital role in learning, as it helps students in understanding their progress and stimulates improvements in their work [77, 81].

\ I5: I’m all for giving students as much information as they can use…and you know…they can use [analytics] to look at their progress.

\ I1: Yeah, I would love students to explore more zoom levels…because usually, I think it is more like…I see it as an overall picture…but they don’t really utilize being able to kind of go in to certain areas or zooming in to certain parts and elaborating…[Also,] maybe spatial clusters just so that they could be more aware about how they separate.

\ While both I1 and I4 advocate for providing students with multiscale design analytics, they also caution against enforcing a specific type of visual organization. According to them, the goal of providing analytics would be to help students to reflect and effectively use multiscale organization, not to have a specific number of scales or clusters across scales.

\ I4: I wouldn’t want them all to look the same like you don’t want to go somewhere and see every painting looks the same, but it was almost as if some people were painting with boards and nails and hammers versus paintbrushes and paint. They just didn’t really get what they’re supposed to be putting on the [multiscale design]. So then it was just like not as effective.

\ I1: [While] they have to become a little bit more mindful of [space]…just seeing how they lay out everything themselves…I would rather not control whether intentionally or unintentionally at all how they see spatial clusters.

\ \

:::info Authors:

(1) Ajit Jain, Texas A&M University, USA; Current affiliation: Audigent;

(2) Andruid Kerne, Texas A&M University, USA; Current affiliation: University of Illinois Chicago;

(3) Nic Lupfer, Texas A&M University, USA; Current affiliation: Mapware;

(4) Gabriel Britain, Texas A&M University, USA; Current affiliation: Microsoft;

(5) Aaron Perrine, Texas A&M University, USA;

(6) Yoonsuck Choe, Texas A&M University, USA;

(7) John Keyser, Texas A&M University, USA;

(8) Ruihong Huang, Texas A&M University, USA;

(9) Jinsil Seo, Texas A&M University, USA;

(10) Annie Sungkajun, Illinois State University, USA;

(11) Robert Lightfoot, Texas A&M University, USA;

(12) Timothy McGuire, Texas A&M University, USA.

:::

:::info This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license.

:::

\