Syllabus

Narrative in Augmented Reality Studio
Fall 2017
Mondays, 9:30am-12:15pm
Credits: 4
Professor: Owen Bell, oab243@nyu.edu
TA: Corey Bertlesen, cb3628@nyu.edu
Course Description
Augment reality [AR] is having a moment right now. With the recent release of technology like Google Tango and ARCore, Microsoft Hololens, Apple’s ARKit, and with the promise of Magic Leap on the horizon, the space available to work in the medium is the largest it has ever been.
The public interest in the future of AR is also at an all time high. The success of projects such as Pokemon GO and Snapchat’s Lenses have caught the world’s imagination and created an audience eager to see more AR content.
Despite all the recent changes, though, the potential of AR remains largely unexplored. We understand the technological possibilities of AR, but what does all this let us do artistically? That is where we come in.
The contents of this course are framed within the context of the Living Stories Through Technology research project, which is investigating how augmented reality can be used for storytelling. Together, we will be exploring what is possible within this new space and how we can leverage its unique capabilities to create new ways of telling stories.
Students in the class will be using local AR techniques such as computer vision, marker scanning, and depth-based tracking to create works that expand, subvert, or alter objects and space to create narrative experiences. Through a process of experimentation and iteration using these technologies, we will begin to develop an idea of how these technologies may be applied to develop the creative possibilities of this new medium.
This is a project based class.  Although many AR techniques will be discussed during the class, the focus of the class will be on ideas. Evaluation in the class will be based not just on your technical competence, but, more importantly, on how you apply your AR know-how. Your projects should reflect a desire to experiment and try to figure out innovative ways to apply AR to storytelling.
This course is based on Mark Skwarek’s Mobile AR Graduate Class and is a modification of Skwarek’s work and Syllabus.  Some of the concepts and ideas covered in this course were developed by Skwarek in the AR Graduate Class and the Mobile AR Lab which Skwarek directs.


Course Objectives
Students will:
  • Learn about the history, current state and future possibilities of augmented reality and mobile technology
  • Develop AR experiences with mobile devices using the latest technology
  • Work on games and other interactive experiences at the service of research
  • Document the process of creating their work
  • Think critically about the technology and push the field forward, as a result of association with the Living Stories research project
Course Structure
The class meets once per week. The session is devoted to a brief discussion of the weekly readings or class critique of the homework due that day, followed by tech tutorials. The TA and, if need, instructor will also be available during office hours to provide support and guidance to students in the development of their projects.
This is a project-based class. During approximately the first two thirds of the semester, students will work alone and in groups to learn about different AR technique and create prototypes exploring how they might be used to tell stories. In the final third of the semester, students will produce a final project that applies the skills they have been developing and further explores the potential of AR as a medium.
Students must come prepared to work and remain focused throughout the entire class. They are also required to complete homework for each and every class without exception.
At the end of this class, students will be able to think critically about what unique artistic capabilities AR provides. Students will gain a strong understanding of the AR industry’s past, present, and especially its future areas of opportunity. This knowledge will let students create new AR works that push the field forward. AR is starting to become more present on the public’s consciousness and today’s artists and developers will be the ones to shape the way the public understands it.
Requirements
The class is open to students across Tandon and Tisch, both graduate and undergraduate. Basic programming experience and familiarity with Unity is required, but advanced skills are not required. Experience with 3D graphics software will also be useful, but is not needed. Techniques such as the appropriation of pre-made models will be used to streamline the project pipeline. This will allow students to focus on their concept instead of getting bogged down in the content creation process.
This is a project based class. Students must finish an AR project that is at a graduate level. If technical problems occur students are expected to find solution to work around them. Technical issues are an inevitability of working in a new technology space and your ability to handle them is part of your grade in this class. That said, the point of this class is the exploration of ideas and concepts, rather than a demonstration of technical excellence. Making sure the instructor aware of issues you encounter will give you the possibility to find ways around these problems together. Students that are upfront about problems they encounter and are proactive about finding ways to navigate them, whether by technical changes, adapting their concept, or other solutions, are far less likely to see their grade reduced should a tech problem derail their project. We are learning about AR together, do not be afraid to come forward when something is not working, because chances are the instructor will learn something from this too.
  
Class Attendance
Attending and arriving on time to all class sessions is required and expected. This includes all labs, recitations, and critiques. If you will be missing a class due to illness, or unavoidable personal circumstances, you must notify your professor in advance via email for the absence to be excused.
Unexcused absences and being late to class will lower your final grade. Three unexcused absences lower your final grade by a letter.  Each subsequent unexcused absence will lower another letter grade. Two tardies will count as one unexcused absence. Arriving more than 15 minutes late to class will also count as an unexcused absence.
Assignments are due on specific dates; extensions will only be granted in exceptional occasions. Late assignments will be discounted a 10% of the total grade for each day it is late. This discount can be the difference between a B and a C; therefore the deadline should not be taken lightly. Unless there are extenuating circumstances, no assignments will be admitted after a week of the original deadline.
Assignments
Students are expected to prepare for each lecture by reading the assigned readings for the day and having their homework ready. Tutorials will be covered in class, though students are expected to do in-depth development supported by a TA.
Homework is due at the start of class, students will receive a homework grade every class. Homework should be uploaded to the class blog before the lecture session; work uploaded after the start of class will be counted as a zero. Turning work in late will not modify the zero. The assignments must be ready for group critique at the beginning of the class. For the final projects, students must show progress on their projects every week.
Students must make sure they have access to the software needed in the class. Students MUST have access to Photoshop and Unity outside of class. If the students have technical issues installing the software reach out to the TA or instructor. Having consistent access to a computer running this software is a requirement for the course.
Weekly Homework
The first two-thirds of the course will consist of short assignments that will allow students to get familiar with the technology they need to use. Each assignment will be presented for critique by the rest of the class the day that it is due.
All assignments must be documented through photos, video and text. Documentation is critical to developing AR as many projects are site specific or based on rapidly changing technology. This documentation counts as the submission for each assignment, and must be posted on the class blog before the class period that it is due.
The assignments are described below, and will be graded on originality and use of the technology, as well as on the quality of the documentation of the development process.


Final Project (Team assignment)
The final third of the semester would be the development of the final project. The Final project  needs to take place within the requirements of the Living Stories through Technology project. This means you will be using either some form of AR covered in this class, or, if you receive instructor approval, another form of technology you choose to learn yourself. Projects must demonstrate an engagement and understanding of the narrative concepts covered in the course and an exploration of how they may be applied in AR.
Final Project Documentation
Students must produce a final project that will exist as professional documentation, as well as contributing to advance research. For this purpose, students must document their process carefully, both through keeping detailed development diaries as well as a final video. The video should look professional, voiceovers and text and needed if something is unclear. Title and credits [the class name/ etc] should be in the video. The documentation is the largest part of your grade so save substantial time to document; recording demo is a must. The documentation should have a narrative flow, leading the viewer through the project and providing a clear understanding of what the project is and how it works.
Grade Criteria
Class participation is 10% of the final grade. Students are strongly encouraged to participate in class discussion and critiques. Disruptive participation will be factored into the student’s grade grade.
Each homework assignment, as well as the final project, will be graded on the application of the concepts and technology discussed. Another aspect that will be evaluated is how students manage their final project and match their development schedule, based on the periodic peer reviews and meetings with the instructor and teaching assistant.
Grade Breakdown
Class participation and in-class preparedness       10%
Weekly homework                                                          50%
Final Project 30%
Final Project Documentation 10%
Counseling and Wellness
Your health and safety are a priority at NYU.  If you experience any health or mental health issues during this course, we encourage you to utilize the support services of the 24/7 NYU Wellness Exchange 212-443-9999.  Also, all students who may require an academic accommodation due to a qualified disability, physical or mental, please register with the Moses Center 212-998-4980. Please let your instructor know if you need help connecting to these resources.
Living Stories project
This class is made possible because of the Living Stories Through Technology project which seeks to explore the storytelling potential of AR. All work created as part of this course can be part of this research and could be featured as part of talks, papers, and other presentations discussing the course. For any work you do for this course, you can choose if you want to make that work available to be featured as part of this research. If you do not want some or all of your work to be part of this research, that is fine and you can still take the course. Just make sure the instructor is aware of your preferences.
Course Schedule
Week 1, Sept 11th: What is augmented reality today & an introduction to visual storytelling
We will cover the syllabus and a basic introduction to the course. Additionally, class time will be devoted to exercise to get us thinking about the use of space and objects to tell stories.
Week 2, Sept 18th: Spatial narrative & immersive theater
Professor Clara Fernandez-Vara, a member of the Living Stories project, will give a guest lecture discussing basic concepts of narrative and how we can start creating our own experiences using them.
  • Homework: Find an example of telling stories with space. Movies, plays, TV shows, pictures, games, anything is acceptable. Be ready to present on the piece and break it down for the rest of the class next week.
  • Reading:
    • Game Design as Narrative Architecture, by Henry Jenkins
    • Game Spaces Speak Volumes: Indexical Storytelling, by Clara Fernandez-Vara
Week 3, Sept 25th: Augmented reality past, present, and future
We will take a general overview of AR technology as it has existed and where it is going. We will also spend time talking about the potential these technologies, past and future, have for telling stories.
  • Homework: In the same groups that you formed during class, return to the location you found or pick a new one. Document the place thoroughly with photos and videos. Create a storyboard describing an intervention in that place, either adding a narrative that was not otherwise there, or subverting one that was already in the location.
  • Read:
    • A Survey of Augmented Reality, R. T. Azuma
    • Technology and the Evolution of Storytelling
Week 4, Oct 2nd: AR Intervention in the Physical Space
It’s time to start taking some of the techniques that we have been learning, and think about how we can apply them to the real world. Students will form groups of two or three and scout out locations on and around the NYU Brooklyn campus that are potentially good options for augmentation. The end of class will be used to discuss these locations and your plans for them.
  • Homework:


Week 5, Oct 9th: Fall break
Week 6, Oct 16th: Marker-based tracking using Vuforia
Now that we’ve found a location we want to augment, it’s time to start thinking about how we will actually augment it. Class time will be dedicated to discussing the Vuforia plugin for Unity and exploring how we can use it to augment spaces by tracking markers. The remainder of class time will be dedicated to creating experiments using Vuforia and getting used to its capabilities.
  • Homework- With the same groups as last week, return to where you planned the intervention, and create it using Vuforia. We will be experiencing these as part of class week so make sure that you have tested them on location multiple times and accounted for possible complications. Be aware that aspects of a location can change rapidly, so counting on having a particular marker available to scan may not always work. Make sure that your projects have solutions for tackling these issues.
Week 7, Oct 23rd: Sound and AR
While we tend to think of AR as a visual medium, sound is just as important. Even a small amount of audio can dramatically enhance an experience. This week, we will spend time looking at uses of AR with audio and learning basic techniques that will let us add sound to our experiences.
  • Homework: Building on the work from last week, use the audio concepts we discussed in class to expand your location works.  You can use audio to either support work you have been doing so far, add new content, or even subvert your previous work. I encourage you to play around with and explore different possible ways of adding sound to your projects. React to the feedback you received on project and use sound, as well as changes to the rest of the content to improve on your work.
Week 8, Oct 30th: Depth-based AR
The most exciting new developments in AR in the last few years have been those using computer vision to identify objects in and the layout of the world around us. This technology can allow us to create experience that react to the world around us and behave as those they are physically present.
  • Homework: Form new groups of two or three. Pick a location at MAGNET and plan a game or experience in that place using depth tracking AR. Create storyboards and mock-ups of your plans for this project and create a small interactive proof of concept. Be thinking about what you may be wanting to do for your final project and how you can explore some of those concepts with this assignment.
Week 9, Nov 6th: Further concepts in depth-based AR
  • Homework:
    • Complete AR project from previous week
    • Prepare pitch presentations for final project
Week 10, Nov 13th: Final project pitches
Class this week will be dedicated to critiquing the projects from the previous week as well as seeing everyone’s pitches for the final project and providing feedback. Each student or group will pitch their idea for the final project and receive feedback from the professor and class. If a project is approved, students are free to begin work. In the event that more development is needed on an idea, additional time during or outside of class will be schedule with the professor.
Week 11-13, Nov 20th-Dec 4th: Critique and group work on projects. Additional topics as requested by the class
Week 14, Dec 11th: Presentation Rehearsals
The final presentation is a large part of your grade as well as being important for documentation on the Living Stories project. All class time this week will be for seeing each project’s presentation and critiquing it. No part of your final presentation grade will be attached to this rehearsal, it is purely for helping you give the best possible presentation the next day. This does not mean that the rehearsal is optional. Failing to present will count against your class participation grade and will mean you have no feedback on how to improve for the final presentation the next day.
Week 14.5, Dec 12th: Final presentations

All groups will present their projects. Each one will be recorded for documentation of the Living Stories project, excepting students who do not wish their work to be used for the project, and guest critics will provide feedback.

No comments:

Post a Comment

Subscribe to: Posts (Atom)