Team: 4 members | Date Completed: December, 2018
Overview
HeirLüm is an Augmented Reality application that allows users to capture and share location based music experiences.
Preview: Application Demo
|
The Challenge
The prompt for this project was to re-imagine the current music listening experience. This problem space offered a unique set of challenges, because it required my team and I to re-define a successful technology that people are already comfortable using. During the primary stages of ideation, our team realized that, although meaningful in its own way, the current methods of listening to music were highly individualistic and could result in isolation. This method of interaction also encourages users to be fastened to their phone screens, leaving them completely oblivious to the landscapes and the people that comprise the places that they call home. Our primary motivation when creating HeirLüm was to create a music listening experience that encouraged users to interact with and contribute to the community and the environment that surrounds them.
Our motivations first manifested in a dynamic sculpture that when touched would reveal a recorded and uploaded video of a performance that occurred in the area that the sculpture was located. While we were satisfied with this initial approach, our team realized that the sculpture's limited and localized influence prevented our product from effectively representing the culture and landscapes of everybody who listens to music. Building upon the interactions and the ideas that we defined in the first iteration of our product, we decided to create an AR application because we felt that we could better achieve our goals without the confines of a stagnant installation.
Our motivations first manifested in a dynamic sculpture that when touched would reveal a recorded and uploaded video of a performance that occurred in the area that the sculpture was located. While we were satisfied with this initial approach, our team realized that the sculpture's limited and localized influence prevented our product from effectively representing the culture and landscapes of everybody who listens to music. Building upon the interactions and the ideas that we defined in the first iteration of our product, we decided to create an AR application because we felt that we could better achieve our goals without the confines of a stagnant installation.
|
First Iteration |
|
The first iteration of our project was a music sculpture that allows city-dwellers to interact with, explore, and discover the musical memories of the city that they live in.
Analyzing the Current System
We began our ideation process by identifying the methods and motivations of the current music listening experience. Our team focussed on identify why people listen to music, how current music experiences work, and what forms or platforms that people use to listening to music. From this brainstorming came a number of valuable insights:
Why
How
What
Why
- To share a moment, memory, or emotion.
- To express culture and individuality.
- To inspire.
How
- Recorded sounds and written lyrics.
- Live performances (on the street and at venues).
- Rhythm, rhymes, and complementary chord changes.
What
- Spotify and other online music streaming services.
- Cd's, records, cassettes - physical artifacts of recorded music.
- Portable listening devices - headphones, phone speakers, car speakers.
This initial brainstorming session helped our team identify the aspects of listening to music that we wanted to highlight in our redefined listening experience:
- People take pride in finding and sharing music.
- Music can have an element of nostalgia that is connected to time.
- Music can be a form of cultural expression.
Ideation
My Role
During the ideation phase of our design, I created one of the storyboards that helped facilitate exploration into the idea of what a new music listening experience would look like. During the early stages of design I also facilitated and managed brainstorming centered around the aspects of music listening listed above, and make sure to keep the team constantly thinking about the end-user of our product.
It was this storyboard that inspired the idea of an Augmented Reality music listening experience that highlights memory, past experiences, and location. We had one main goal while designing our music listening experience - to keep people off of their phones. The last thing that we wanted to do was to encourage people to constantly look down at their phones, aimlessly walking through the beautiful cities and sceneries in which they reside. At this point in our design process our team realized that, whatever form our product would take, we wanted it to encourage users to be more mindful and curious about the landscapes that surround them.
Mission Statement
My Role
Helped facilitate discussion about our team's fundamental goals and motivations for creating a new music listening experience. I kept the team on track while we were creating a discrete mission statement. Also assisted with the articulation of our team's mission statement.
"How might we immerse people in the musical culture of a city or environment through exploration and spontaneous discovery?" |
We had four main goals that we kept in mind while drafting our mission statement:
- To engage pedestrians by offering subtle yet meaningful interactions with the city they live in.
- To encourage the exploration and pursuit of music performances and the surrounding environment.
- To bring people together in a meaningful way.
- To emphasize the culture of the music that people are listening to.
Thought Association Diagramming
My Role
Contributed to the thought association diagram and suggested the spontaneous discovery and exploration ideas to our group.
Process
We started by defining the big-picture ideas that came to mind when we thought about listening to music, and we wrote those down at the top of the page. Then, we started to break down those central ideas into smaller, more distinct terms that we identified in the analysis described above. After having a visual representation of the salient topics of listening to music, we began to visualize what we wanted our experience to represent. We wrote down main goals such as exploration, spontaneity, and discoverability. We also wrote down some existing technologies whose unique public engagement inspired us, like geocaching, Jack Mackie's bronze footsteps in Capitol Hill, and even easter egg hunts.
Sketches: First Iteration
My Role
Helped consolidate ideas from storyboarding, initial ideation and inspirations from the previous steps into one cohesive experience.
Process
Our first ideas focussed on time, the passing of time, and how music and memories are intertwined with time. The form of our first sketches closely resembled a jukebox, as we thought this skeuomorph would help people recognize our structure as a musical artifact that they could interact with while also instigating memories of interacting with jukeboxes with friends. This iteration of HeirLüm involved people walking up to the installation, using a lever shaped like a music note to select a time period and discover a musical performance from that era via speakers and a projection.
Strengths
Weaknesses
- Engaged the public in a way that allowed people to remember or discover music from the timeline of musical history.
- Placed an emphasis on the relationship between music and time.
Weaknesses
- Did not encourage the exploration of the environment surrounding the artifact.
- Only one person could use it at a time, it did not support meaningful interactions between people.
- The affordances of the artifact were not obvious; the form of the jukebox did not tell people what it does or how to use it.
Sketches: Second Iteration
My Role
Provided the idea of having "branches" that extend around suburban blocks, thus extending the discoverability and community engagement of our installation. Drafted initial sketches that lead to this final sketch, created by my teammate.
Process
The second iteration of HeirLüm build upon the strengths from the first version of our project. We increased the number of interaction points to place an emphasis on the discovery, pursuit, and exploration of musical memories throughout people's surroundings. This version of HeirLüm consisted of an interconnected network of nodes which displayed small snippets of performances that took place around the area that the artifacts were installed. When touched, these previews would pulse away along the structure in the form of light, directing the person interacting with the structure to pursue the musical memory. This pursuit would terminate at the heart of the sculpture, where the musical performance they were following would be projected and the sound would be played through speakers.
Other Initial Ideas
Strengths
- Engaged the public and promoted interaction between people interacting with the artifact.
- Reached a broader audience.
- Encouraged discovery of a person's surroundings.
- Limited by the physical structure of the artifact. Would be fragile and the material of the artifact would be expensive.
- The experience was confined by the amount of physical space the installation reached.
- Store owners, residential buildings, and government facilities might not approve of having our installation being on the walls of their property.
|
Experience Demo #1 |
|
|
Key Interaction Moments
Reflection
Our team was very pleased with the interactions and experiences that we created for the first iteration of HeirLüm. However, we felt that there were still some issues that prevented us from creating the accessible experience that we had envisioned.
To tackle the limitations and boundaries of a physical and fragile sculpture, we decided to transfer the experience of HeirLüm to Augmented Reality. A big motivation for our team was to encourage people to stay off of their phones while promoting the exploration and discovery of their surroundings. For these reasons, we felt that a line-of-sight display would be the perfect platform to create the music listening experience that we were striving towards.
- The sculpture provided an experience that was only available to the people who were physically near them. We did not want the experience of listening to music to require somebody to be bound to a specific geographic location.
- We also understood that the physical structure of our sculpture would be hard to maintain, as it is fragile and expensive. Our sculptures would also require the approval of the cities they were located in and of every property owner that our sculpture spanned across.
To tackle the limitations and boundaries of a physical and fragile sculpture, we decided to transfer the experience of HeirLüm to Augmented Reality. A big motivation for our team was to encourage people to stay off of their phones while promoting the exploration and discovery of their surroundings. For these reasons, we felt that a line-of-sight display would be the perfect platform to create the music listening experience that we were striving towards.
|
Second Iteration |
|
A New Approach
At this point, we decided to take a step back to analyze our project holistically. We looked at how effective our sculpture was at achieving our goal of immersing people in the musical culture of their city via exploration and spontaneous discovery. From this analysis, we realized that the experience we were designing was constrained by the physicality of our sculpture, and it did not truly encourage the exploration of people's surroundings. We decided that HeirLüm could better achieve our goal on the Augmented Reality platform.
Right away, we realized that we did not want to just transfer our sculpture into AR. We wanted to redefine the interaction model and interface of HeirLüm to take advantage of the new opportunities that the Augmented Reality platform provides. For the rest of the project, we constantly asked ourselves "Does this interaction/feature demand that our application be in Augmented Reality?" If the answer was no, then why were we putting it in an AR application?
Right away, we realized that we did not want to just transfer our sculpture into AR. We wanted to redefine the interaction model and interface of HeirLüm to take advantage of the new opportunities that the Augmented Reality platform provides. For the rest of the project, we constantly asked ourselves "Does this interaction/feature demand that our application be in Augmented Reality?" If the answer was no, then why were we putting it in an AR application?
New Challenges
Redesigning HeirLüm for Augmented Reality proposed a unique set of challenges. The biggest challenge was figuring out how we should provide directions for the people using our technology. Should we leave a "trail of breadcrumbs" that a user would follow to the HeirLüm they were pursuing? Should there be a line that is projected on the ground (similar to Google maps) that provides directions? We were unsure of how much we should "hold the hand" of a user while directing them from one place to another. Another notable challenge that we faced was how we should handle the content of our application. How did we want to filter and represent the content that is supplied to our application? These are questions that we had to answer in order to make an AR experience that effectively achieved our goal.
Navigating Through the World
We began work on this new iteration of Heirlüm by sketching out different approaches towards directing users through the real world. We initially drafted a "breadcrumb" navigation system that would provide a trail of music notes that a user would follow to the HeirLüm they were pursuing.
Strengths
- Offers a clear pathway for user to follow to the HeirLüm they are pursuing.
- Prevents people from getting lost or from straying off-course.
- Does not promote exploration, but rather encourages people to simply follow directions from one place to another in the most direct path possible.
- Is a visual burden for users, and could detract from user's surroundings.
Reworking the Navigation System
My Role
Had the Idea of a compass that surrounded the feet of the user. Sketched, mocked up, and designed the final navigation system for our final experience demo.
Left: My first sketch of the HeirLüm AR navigation system. Right: Two different views of our final AR navigation system that I created.
|
We did not want the navigation system of our experience to be too prominent in the line-of-sight of our user. Our team wanted to encourage discovery and exploration, not somebody following around a trail of crumbs or walking along a big blue line. We chose to provide a "compass" around the feet of the user, because it encourages them to take their own paths to the HeirLüms that they are pursuing. If a user wants to explore and forget about the direction they should be going in, they simply can avoid looking at their feet, but if they need guidance towards their destination, they need only to look down for a subtle yet effective reminder of the direction they should be heading towards.
Defining Interactions
My Role
Contributed to the ideation of the interface and interaction flow our final product and created sketches that helped my team and I create the final interface. Advocated for inclusion of the "discover" menu, which directs users towards a random HeirLüm that was dropped near by.
Left: An initial sketch I made of a potential interface for our product, showing the menus and information about each HeirLüm that should be displayed. Right: A hi-fidelity mockup that refined our initial sketches. This interface is close to what we used in the final experience demo for our project.
|
We were unsure of how much information we wanted to show users when they were navigating through our product, but we decided to keep the interface simple by only offering interactions that were necessary. Our team felt that it was important to keep a line-of-sight display simple, because anything too complex would distract users from their surroundings and could prevent our users from being aware of their surroundings. We also felt that a complex interface would prevent our users from enjoying their surroundings.
Method of Interaction
Left: A sketch that shows a method of interacting with the AR interface. Right: A 3D printed prototype (By Erfan Dastournejad) of the interactive ring that we used in our final experience demo.
|
We wanted the interaction method to be physically oriented instead of gesture oriented, because users would probably be using our product in crowded areas (concerts, street performers, local shows, etc). Using hand gestures to interact with our product would be limited by the amount of physical space and lighting that a user is in, and these are two resources that are infringed upon in the use context of our project. For this reason, we created an accompanying ring that users would wear to scroll, select, and record in HeirLüm. Scrolling up or down on the ring is reflected in our AR display, and tapping on the ring will select, record, or drop an HeirLüm depending on the menu that the user is on.
|
Experience Demo #2 |
|
My Role
Contributed to script writing, scene planning, and posed for / narrated the experience demo.
Planning
We started creating a final experience demo by ideating a storyboard that defined the shots, angles, and protagonists of each scene that we created. My teammate, Erfan Dastournejad, created a storyboard that incorporated the team's ideas and goals. With this we went to the streets to shoot our experience demo. We used this demo not only to describe how people will interact with our product, but also to display how seamless and nonintrusive our reimagined music listening experience was. We also tried to point out the key interaction moments that we designed for, and how these moments might come about. My teammate Nikita Kovalovs then edited together the experience demo that is presented below.
Storyboards that our teammate made planning out the scenes for our final experience demo. These storyboards consolidated the planning and ambitions of all of our team members, and were followed closely when shooting the final demo.
|
Final Deliverable
|
|
Reflection |
|
This was my first deep-dive into designing for Augmented Reality. Designing for AR presented me with unique challenges that I had never before encountered, but it also allowed me to adapt and apply my understanding of the UCD process to an entirely different problem space. I enjoyed working on this project because my team and I pursued an untraditional approach to redefining the music listening experience. As the final deliverable for this project was an experience demo, we were not limited by the physical laws of nature, nor were we forced to simply "find a problem and solve it". This project taught me how to hold myself, my designs, and my team accountable in a boundless design space, and that is a lesson that I will carry with me through the design career.