In early 2020, Leah Scolere, assistant professor in the Department of Design and Merchandising, was assembling a team of students and faculty from the Interior Architecture and Design, Computer Science, Sociology, and Electronic/Digital Arts Programs to create an augmented reality (AR) app to highlight Colorado State University’s Nancy Richardson Design Center (RDC) as a “building that teaches.”

Soon after the project launched, COVID-19 forced the collaborators to work from home, the RDC sat mostly empty, and the team had to take on a new challenge: how could this project continue through the isolation of the pandemic? After some delays, the team engaged in creative problem solving, and the app is now ready for you to test the next time you visit the RDC.

As part of grant-funded research by the American Society of Interior Designers, the AR app assesses how buildings, specifically the RDC, can enhance the learning, creativity, health, and well-being for occupants. This is particularly relevant as the RDC meets LEED-Gold standards for sustainable practices and the WELL building standards for improved human health and wellbeing.

Scolere and Laura Malinin, associate professor in the Department of Design and Merchandising and director of the RDC, worked with an interdisciplinary team. Francisco Ortega, associate professor in the Department of Computer Science and Jeni Cross, professor in the Department of Sociology and research director for the Institute for the Built Environment helped pilot the experience.

The app’s mission and goals

Post-occupancy evaluations, or POEs, allow building occupants the opportunity to assess if a building meets their needs. POEs are usually a simple survey, but Professor Scolere’s team posits the AR app will be more effective by including questions in the context of the physical building.

The AR app will go beyond a building evaluation and educate users on the features of the space and their use. The technology enhances the experience of the space by overlaying digital information in real-time on the interior environment.

“This means that as a building occupant moves their phone around the physical environment, they will see in real-time digital overlays such as audio, video, 3D models, and images on the physical space,” Scolere explained.

The app shows the user the features of the RDC that make it a “building that teaches.” For example, the app highlights features of the building to encourage occupant well-being including areas for collaboration, mental restoration, and opportunities for physical activity along with informing users of specific building materials used within the RDC.

Overcoming challenges with virtual collaboration

As the project was gaining momentum, the researchers encountered challenges due to the COVID-19 pandemic. Will Schmidt, a senior in electronic arts, led the creation of the 3D assets for the app.

“We couldn’t meet in person, which was one issue, but another big one was not being able to go to the building we were going to give a tour on,” he explained. “We had the 3D design plan for the building, but there’s nothing like walking around the physical space.”

Meghan Jackson, who graduated with her degree in interior architecture and design, agreed.

“Brainstorming how the augmented reality app and the physical environment could be in conversation with one another was very challenging to do from a distance,” Jackson said. “It took longer to generate ideas, create graphics, and program those graphics into the space without being able to physically be there, so many of our computer programming steps took place after classes began in person again in Fall 2020.”

The team overcame challenges with communication between the technology and the physical space along with collaborating virtually. They got through it with many teams/zoom calls and increased their frequency to meet deadlines. Additionally, coming from a wide range of disciplines, sometimes team members had to rely on industry-specific jargon that the rest of the group didn’t understand. To develop shared project language, the interdisciplinary team created a range of digital visual tools to help communicate their ideas and progress virtually including scene demos, digital storyboards, and mock-ups.

“The main takeaway I had from working on this project was the importance of clear communication in team projects, especially cross-disciplinary,” said Jackson, “Making sure we were all on the same page involved learning a fair bit of terminology from other fields.”

The technology behind the app

While the team was stuck at home, they utilized a Revit model created by the designers of the RDC to “virtually see the spaces we were making augmented reality graphics for,” said Jackson. Revit is a 3D building information modeling program that shared floor plans and renderings of the RDC for the team’s reference.

To communicate the flow of the app, researchers created a digital storyboard to provide a basic outline for each of the areas of the AR design. The storyboard included floor plans from Revit, pictures of the spaces in the RDC, and the POE questions for each area.

Using Revit and the digital storyboard as a starting place, the team then built the app prototype. Crispin Haro, a senior in computer programming, created 3D objects with Unity Game Engine.

“Each object has its own C-sharp script that lets us control its behavior,” he said.

One of the most rewarding experiences for Haro was learning about Placenote SDK, software used to make augmented reality and virtual reality more attainable for the creator. The technology is something that Haro hopes to continue using in the future both in work and as a hobby.

Get involved in the project

The app is now in the stages of beta testing and could use your help.

“If students, faculty, or staff that use, visit, or occupy the RDC are interested in getting involved, they can submit their contact info and our team will reach out to set up a time,” said Scolere. “The experience involves testing the app and giving feedback along with an informal conversation at the end to learn more about their experience.”

You can reach Scolere at leah.scolere@colostate.edu to learn more. The project earned high honors for the 2021 Celebrate Undergraduate Research and Creativity Showcase at CSU. View the team’s research poster in the CURC virtual platform.

Jackson highly recommends getting involved in research as an undergraduate. “My involvement with this project began by going to my professor’s office hours and expressing my interest and has since evolved into me working on three different studies. These have taught me so much about collaboration and the design industry . . . I highly recommend getting to know your professors—you never know what conversations and opportunities it can lead you to!” she said.

Scolere’s research lab has many opportunities for undergraduate and graduate students to help with research. You can visit the Connected Environments Research Lab to learn more about current projects.

The Department of Design and Merchandising is part of CSU’s College of Health and Human Sciences.