Short distance from idea to project

Seed funding enables lecturers to test new methods in their teaching.
- The mini-projects contribute to connect the lecturers on five different campuses at two universities that make Excited more closely together.

These words come from Robin Munkvold, who is in the Excite management at Nord University. Girts Strazdin's associate professor at the Department of ICT and Science in Ålesund who has received seed funding is of the same opinion.

- Excited mini-projects are a very good form of support - there is little paperwork and administration for the lecturers who run projects. For example, one does not have to spend a lot of time and effort on the format of the application. I find it motivating that there is a very short path from an idea to a project.

Learning through Construction (LtC) is a teaching method that is widely used in IT education, and the students learn skills and theory through practical projects. This form of learning has its own core-project under Excited. In the autumn of 2017, the core-project started to map all courses under the center that utilize LtC in their teaching. In the next step, all relevant lecturers were interviewed about their approach to LtC in their teaching.

- The interviews gave us an entry point to discuss teaching methods and how we work and think about teaching at the various institutions, campuses and subject areas, says Robin.

One of the lecturers who participated in the study was Girts Strazdins in Ålesund.

- We talked about the subject I teach with a colleague, System development and modeling. During the conversation, I heard about GameLab and the methodology they used at Steinkjer. The conversation inspired me to try out the method in my own subject. We were already running development projects in the subject, but it was clear that the system to be developed did not arouse the commitment of the students.

Girs found that two things had to be in place to make the project teaching engaging for the students: 1) real customers, who are interested in the end product; 2) the projects had to be changed to involve game programming. - I got lots of important suggestions and help from Robin on how I could handle technical things such as game frameworks, meetings with customers and usability testing. In addition, it was also important to get inspiration and be able to discuss things that came up along the way with a subject teacher who had completed the teaching method with good results before.

Once the idea had developed, Girts applied for seed funds from Excited in the form of a mini-project. Part of the support was used for two student assistants in Ålesund as support to program a game development tutorial for the students.

- All groups managed to deliver a playable product by the end of the semester. The students experienced a realistic scenario: real customers with real demands, real problems such as customers not being sure what they really want, and changing their mind in each meeting. They faced real development environments and tools like those they will meet in their work-life after graduation. The students also reported after the exam that these were the two major motivating factors for them in the course. External customers, and that they were allowed to develop games.

"Excited has been an important network for me - I have received support for the development of my teaching in several ways: contact with other colleagues with experience, arena for the exchange of ideas, and not least financial support in the form of the mini-project." I definitely envision that I will apply for more projects in the future.

Final report on the project

Poster presented at UDIT 2018

Excited mini project, connected to Excited projects Learning through construction and Sharing and diversity

Teachers involved:

• Kristiania Universtiy collece: Trygve Bjellvåg, Ole Goethe
• Nord: Robin Isfold Munkvold and Trond Olav Skevik

Goals

• Develop a rubric or a guideline that is useful and effective for doing Cross-Campus Student Peer Review on Games Development / Products
• Increasing student engagement with the use of Cross-Campus Student Peer Review on Games Development / Products
• Improving assessment criteria on Game dev courses (or on IT-development courses in general)
• Improving student engagement through the use of Cross University / Campus Student Peer Review
• Broaden the students’ perspectives on their work, strengths, and weaknesses by sharing their work with students from a different background and setting
• Creating a student active cross-campus activity
• Enhancing the understanding of such a pedagogical method and developing a process and tools for long term gain.

Cross Campus Student Peer Review Project

When students are working on Game Development Projects, peer review provides them with the opportunity to receive feedback from other developers familiar with the field. This can provide students with more suggestions and ideas for revisions, potentially increasing the quality of their games / projects / products. Peer review also allows students to identify strengths and weaknesses in their own games after having reviewed their peers’ games (and documentation). By structuring peer review, teaching staff can maximize the usefulness of the feedback that students give and receive. Excited can assist in the development of a peer review rubric / guideline and help facilitate this.

Based on previous experience within P4 it is important to have someone local that can facilitate the process in cross-campus learning. In this project we are planning to use TAs at both campuses to help facilitate the collaboration and to gather data and help out on documenting the project.

The project will connect its activities to one course at each campus, run the spring semester of 2020. These courses are:

• Game lab 5 (3rd year Game students at Nord University, Steinkjer)
• Project course at Kristiania

Game lab 5 is the game development project for the 3rd year Games students at Nord, run in their last semester. Game Lab is a work simulator aiming to give the students a controlled environment in which to learn how to work together in a development team. Focus is on game development- process and product. Theme for the game lab is Games for Learning.

Course description, Kristiania

Project Plan Outline

1. Planning the whole project, defining the contexts, contents and project timeline with responsibilities, tasks, etc.
2. The development of the rubric / guideline 
   Making an online solution
3. Preparing workshops with the students
4. Maybe creating a gaming contest between NTNU and NORD students (or similar) - working with students’ ownership to the project
5. Skype meetings with students from both campuses
   Presentations midway through the semester and doing peer reviews
   Presentations at the end-of-term doing peer reviews
6. Gathering data (questionnaires / interview) on the process of the project, looking for ideas on improving both the process and the tools on students’ motivation (effects of doing a cross campus / cross University student peer review)
7. Writing a report and a article / paper based on the project results
8. Planning a new and improved Cross Campus Student Peer Review Project for the fall semester of 2020.

Budget

Most of the practical work will be done by TAs with the support of teachers.

We expect each TA to have to spend about +/- 3 weeks on the project, meaning approximately 115 hours each. In addition, when doing workshops with the students it would be nice to be able to serve simple food and beverages, giving it all a nice social setting at the same time.

With about 25 students at NORD and 25 students at Kristiania taking part in this project and the workshops (3-4 workshops), we are calculating approximately 3.000kr per campus per workshop. With 4 workshops at the two campuses this adds up to 24.000kr for food and beverages

TA salary is approximately 190 kr pr hour, incl social costs. With 230 hours in total for the two TAs this would calculate to 43.700kr.

Total budget (24.000kr + 46.000kr): 70.000kr

Contact person for the project: Robin Isfold Munkvold

Literature

• Create-a-Game Assessment Rubric
• Improving learning achievements, motivations and problem-solving skills through a peer assessment-based game development approach
• Peer review: does it really help students?
• Collaborative learning through formative peer review: Pedagogy, programs and potential

Facts

• Project manager: Helga D. Isfold Sigurdardottir
• Other participants: Female students at Nord University, i.a. Mette Marybelle Sundfær, Bibi Kjartansdottir, Katrin Birgisdottir & Birta Maria
• Start and end date: 18 October 2017 - 1 June 2019.
• Subproject in the Excited mini-project falls under: Projects of Becoming
• Sum awarded from Excited: NOK 50,000

Descriptions og the project

The aim of the project was to form a professional network for the female media technology students at Nord University, campus Steinkjer. as they are in a minority and have a relatively high drop-out rate. The network is led by the female students at NORD and organizes regular events, where students meet, seek inspiration and learn both from each other and from employed role models in the media technology industry.

The network will organize trips to visit local media technology companies and events. The network also aims to ensure that participants are well informed about gender equality in the media technology industry, about their rights and opportunities

The result so far is that the female students appreciate the measures. So far, we have arranged 2 media technology meetings in the evening with input from female role models in the media technology industry. Furthermore, the students have participated in 2 Girls & Technology events in collaboration between Nord University and NHO / Nito.

Evaluation

In the project application, we described that we would use a survey to measure the results of the project. The following questions were listed - see also answers, from October 2018:

• How do female media technology students experience the student life?

In short, the female students thought that student life was on average good and average inclusive, while 4 of the 6 respondents thought that student life was to a high or relatively high degree lively.

• Has the student experience for female media technology students improved through the mini-project?

Half of the respondents answered «yes» and half answered «maybe».

• Do the female media technology students feel empowered by the mini-project?
• Do they experience increased professional confidence, as a result of the mini-project?

The distribution of answers was equal for these 2 questions, where 3 of 6 answered "yes", 1 answered "maybe" and 2 answered "no"

• Do the female media technology students feel that they have sufficient peer support during their studies?

2 out of 6 answered "yes", 3 answered "maybe" and 1 answered "no"

• Do proportionally more female students graduate from media technology as a result of the mini-project?

This is a bit early to measure, but after we got the project, it has at least not seen dropouts of female gaming students in the 2nd or 3rd year - and rather the opposite if anything, then at least 1 female exchange student (from Spain) has applied for permanent relocation to Nord University.

• What have the female students that organize the Nord Girl Network learned from doing that?

“I learned more about organisation and such”

“Through the events I participated in, I could feel more confident to have a dream to work in media industry as a woman”.

“I realized how big the Network actually is in Norway and influential which gave me more contacts within the field”
 

In summary, the respondents are more satisfied than dissatisfied with the project so far. Most of the respondents feel more empowered and more confident in the academic, as a result of the project. This gives us a basis to continue in the same way - at the same time as we make sure to further build up and improve the project.

Quick factual info:

• Mini-project leader: Greg Curda
• Other participants: Robin Munkvold, Line Kolås, Advisors
• Start and end date: August 28 – October 20, 2017
• Related project(s) in Excited: P3, Learning Through Construction
• Funds given from Excited: NOK 15 000
• How funds were spent:
  • Funds were awarded solely to the two TAs involved, Magnus Nordfjord and Jonas Grannes, for their efforts in the following areas:
    • Development and prep for the classes, 10 hrs each
      • Jonas’s development time was already covered under a separate miniproject
    • Lecture/demonstration and lab time, 12 hrs each
    • Feedback sessions on initial assignment, 5 hrs each
    • Grading of final project handins, 5 hrs each
• Own effort: Responsible Instructor has spent approximately 32 hrs outside of normal work time, not paid for by Excited in the following manners:
  • Advise and supervise development of class, 4 hrs
  • Advise and supervise implementation of class, 20 hrs
  • Advise and supervise feedback and grading, 4 hrs
  • Acquisition of post-project data and feedback, 6 hrs
• Related course(s) Nord University FTV2010, Sound 2, specifically tailored to Game Design students
• All data has been ethically collected, and the results can be published on Excited web

Descriptions:

• Objective:
  • The project was designed to gauge the effectiveness of using “peer expert” TAs in the instruction of specific techniques pertinent to the Unity and Fmod game engines within the context of Sound Design for Electronic Video Games. The project was conducted as part of a second level class that was shared with Film & TV students. Given that the two disciplines were quite different in the implementation of sound, the project was designed to bring more specific education relevant to Game Design to a general post-production sound class. The goals included:
    • To gauge the acceptance and validity of instruction in these specific formats, and
    • To gauge the effectiveness of using TAs that were agemates, yet had done substantial research and development in these fields.
• Methodology:
  • Unity class was designed using a step-by-step instructional approach, while the Fmod class instructions were vaguer, to assess motivation of the students to employ independent research to complete their understanding.
• Evaluation:
  • Evaluation was anecdotal in form, consisting of:
    • Student survey form, 14 responses from 19 students
    • 6-member focus group interview conducted by Greg Curda and observed by Robin Munkvold
• Results:
  • Students were mostly positive about the value of the instruction.
    • 100% of survey respondents were satisfied with Unity training, compared to 72% for Fmod.
    • Although most agreed that Fmod was valuable, two main issues were cited:
      • Fmod instruction was not as organized as Unity instruction, and
      • It was discovered that Fmod exhibited compatibility issues with WebGL, the browser-based game testing platform currently in use, thus rendering it difficult to implement in the current context.
  • Students generally were very positive (86%) about the use of the TAs and, from focus group, felt that the TAs may know more about the issues they struggle with as students, but also gave a nod to the importance of the teacher’s experience.
    • “They have had to learn the software themselves not long ago, so they have a better understanding on how to learn to use it.
    • “They are likely to have a similar perspective as students, which makes (it) maybe easier for them to answer questions in a meaningful way.”
    • “… it's the experience and, perhaps more importantly, their experience in 'teaching' that's important.”
  • Approximately 79% of respondents felt motivated to some degree by the studies, but expressed some frustration by the vaguer teaching style for Fmod.
    • “It was fun to experiment and learn the different audio tricks in Unity, that I didn't know I had learned.”
    • “Unity was good and I felt I had a good grasp of what was taught in the lab classes.”
    • “Fmod was alright, but I learned most of the Fmod tools from a separate resource, like a video tutorial.”
    • “(Fmod) Lecture was now (not) very good, but the sample project and assignment given was fun.”
    • “He went too fast through everything and we had no material at hand, like a basic walkthrough of some sort just to learn how to handle Fmod. The result was we had to tend to youtube and the internet in general to figure stuff out.”
  • Students were divided on motivation to research topics further outside of class.
    • “I explored a few of the topics from the classes, but not much more beyond surface level.”
    • “I wasn't too motivated to explore further, due to everything that was happening with other subject on the side.”
    • “… my team did research Fmod a bit and tried implementing it in our project but ran into some problems with it.”
    • “Too much else to do“
  • Students were unanimously (100%) positive about the reuse of such methods in further classes.
• Conclusions:
  • Students’ satisfaction with the TAs (86%) stems mainly from the fact that they can empathize with the students and reflect that empathy in their instruction.
  • Students feel more motivated (77%) when the instructor is enthusiastic and involved. However, they made it clear that they prefer more structured, step-by-step instruction, requiring little if any outside research. However, in focus group, all agreed that they were motivated to some degree to dig deeper, either by curiosity or necessity, with the latter being sharply criticized. Yet motivation for further research generally did not go beyond the boundaries of class time.
• Future and reuse:
  • Student response indicated that this form of “peer expert” instruction was well received and should be continued in the future. However, students seem to indicate that they prefer the step-by-step approach requiring little, if any, supplemental research. Further research is necessary to find out more about this apparent lack of intrinsic motivation. Perhaps sound is too specialized to draw any motivational conclusions, and perhaps a similar project or study within a wider field is called for.
• Quick popular description:
  • The “Peer Expert” TA Concept: Does it really work?

Links and attachments:

• Appendix 1: Survey Responses
  • https://www.dropbox.com/s/10iy5pbye2loiy2/Excited TA Mini-Project Evaluation_v2.pdf?dl=0
• Appendix 2: Focus Group Notes
  • https://www.dropbox.com/s/s6mksjhqirs3df0/The use of TAs in Sound class - focusGroupInterview-Notes-dec6-2017.docx?dl=0
• Appendix 3: Unity Project Sample Clip
  • https://www.dropbox.com/s/teg48srytsl0z3z/Unity.mp4?dl=0
• Appendix 4: Fmod Project Sample Clip
  • https://www.dropbox.com/s/yo61ax907jbvism/FMOD.mp4?dl=0
• Longer reports or papers:
  • A longer paper discussing the impact of using “peer expert” TAs in specific education is planned. The paper will be a case study to explore the TAs effectiveness in enhancing the learning environment by fostering greater motivation, both intrinsic and extrinsic.

Facts: 

Mini-project leader: Babak Farshchian
Other participants: Elena Parmiggiani, Tangni Cunningham Dahl-Jørgensen, Farzana Quayyum
Start and end date of mini-project: January 2019-December 2019.
Related projects: Learning Through Construction,Sharing and Diversity, Career readiness. Note: We have not approached these projects directly but would happily do so if our results are interesting for these projects.
Funds given from Excited: 100 k NOK.
How funds were spent (specified list): We did not spend all the funds1. We applied to have two assistants but hired one (Farzana). In agreement with Excited, we repurposed the 20 k NOK on travel money to travel to SCIS/IRIS in Finland to organize our workshop.
Own effort: Babak Farshchian and Elena Parmiggiani were involved in regular meetings and discussions, organization of the workshop, and authoring the final report. We also have implemented some of the results in our teaching. Tangni Cunningham Dahl-Jørgensen spend a portion of her teaching duty to participate in a literature study,

Objective of the project

"Research methods courses are known to be boring! Few teachers of such courses focus on pedagogical aspects of teaching research method. This is mainly because they teach methods as a by-product of their research. Our project aims to systematize available knowledge in teaching research methods. We want to 1) make a directory of best practices and resources for teachers of research methods, 2) create an online community where teachers of research methods can ask questions and share their experience."

Evaluate and further improve problem-based teaching in the three "Empirical Research Methods in Computing" cources  IT3010, TDT39 and DT8111.

Create a structure for the courses that is scalable so that the courses can be offered to more students at IDI (Master of Technology, PhD, cross-campus) possibly also to other universities in an online version.

Get involved in an international community of research methods teachers.

Results of the project

1 If possible, with the remaining funds we would like to be able to hire a research assistant to continue the study also in 2020.

Systematic survey of literature and existing courses on teaching research methods.
Organization of a workshop on teaching research methods at SCIS/IRIS in Finland.
Draft of a workshop report submitted to the Scandinavian journal of information systems.

A new spin-off study by Elena Parmiggiani about the role of theory in teaching research methods. This study is done in her "pedup" course and will probably result in a journal publication in the future.

Evaluation 

We have not done any formal evaluation of the results so far because all the affected courses are in 2020. We have brought in new ideas into IT3010 and DT8111 based on the systematic survey we did.

We also presented our courses during our workshop in the SCIS/IRIS conference, and compared our approach to other teachers. The feedback from our workshop will be used to create an evaluation framework for our courses.

Future and reuse:
Yes, see above. All the three courses are affected by the results of the project. In addition, we have as a result created an international network of researchers who will be invited as guest lecturers in the future.
We have provided a draft of our workshop report to the Scandinavian Journal of Information Systems. Our goal is that we can use the network and the preliminary results to create a call for paper for a special issue of this journal in the area of teaching research methods.

Links and attachments:
• The community web pages: In these pages you can find information about the community, future events, and pointers to other resources. The pages also include a course directory that is expanding.

• The community project at ResearchGate: You can either follow, or preferably join our project at ResearchGate. We hope to use this as a more interactive forum for the community of practice:

• Zotero libraries:
o Zotero library for literature on the teaching aspects of empirical research methods in IS: This can be a useful resource for those who want to do research on the topic, or see what other researchers have to say.

o Zotero library for building a syllabus for MSc and PhD levels: In this library we intend to collect teaching resources with a Scandinavian flavor. The resources will be available for everyone:
 https://www.zotero.org/groups/2390863/termis-syllabus