A conversation about storytelling, hackathons, indie rock audio, and collaboration with filmmakers and adventurous brands by Gary Shapiro

Gary Shapiro is a writer whose work has appeared in the Wall Street Journal, the Forward, and in the Features section of CreativeTechWeek.nyc.

A 20-year veteran of digital production, Mike Knowlton has always pushed the boundaries of storytelling and technology. He is co-founder and CTO of StoryCode, a group that programs immersive media case study presentations at Film Society Lincoln Center. He led in the production of the first-ever Story Hackathon, an event that was covered by Forbes, The Washington Post, and PBS. And as a creative technologist, he founded and led a number of technology companies including user-interface design firm Nascent State, digital agency BASIK, open-source Flash technology Frontal, and Murmur, a hybrid studio/technology company that pioneers new forms of immersive cinematic experiences. He has led large multidisciplinary teams in developing complex software applications for brands including Bath & Body Works, Gucci, MoMA, Philosophy, Tiffany & Co, and Victoria’s Secret.

GS: You’ve started a number of companies or movements. Let’s start with the non-profit StoryCode, which has held programming at the Film Society at Lincoln Center.  Can you describe what StoryCode does?   

MK: StoryCode is a movement by, and for, immersive media makers.

Over the past five years there have been endless debates about labels…is it is transmedia, interactive, immersive, virtual reality, or augmented reality. From our perspective it doesn’t matter what you call it, since it will sort itself out. But this is an amazing time for innovation in the way stories are being created and consumed.

StoryCode is for people who want to participate in growing this broad and expansive discipline. At the core, all these recent narrative innovations are a blend of storytelling and technology so we named it StoryCode. Its chapters convene real life events designed to educate, inform and connect creators of immersive and interactive stories.         

GS: I’m glad you mentioned the chapters.  I’d like to pick up on that. How did it come to be that there are StoryCode chapters around the globe from Boston to Barcelona and San Francisco to Singapore?  Did you intend from the beginning for this to be an open-source community?

MK: Our initial focus was mainly in New York City and then it just sort of took off and grew on its own. A group of creators from Paris reached out to us and asked about the possibility of starting a chapter in Paris. That got us thinking and we realized the idea behind StoryCode was really best served to be structured as an open-source community. It’s not a traditional non-profit structure, it’s super loose and flexible. Each chapter really has the freedom to develop as they wish and take on the flavor and focus of their own area.

To keep things organized, we have a co-organizer Slack and hold quarterly virtual hangouts. So the organizers get to know other immersive media makers around the world and we can share information that way as well.

GS: You held a Story Hackathon even though the concept of hacking doesn’t really exist in film and creative writing.  

MK: That was exactly our thinking! Take the concept of hacking and insert it into the traditional process of filmmaking and creative writing.

GS: Can you talk about what you did there?

MK: We think of “film as software.” So really what we are trying to do is apply software development principles to storytelling. Things like failing fast, rapid iteration, agile development, etc….We think that’s a better process given how important technology is in these new types of stories. 

Amazing storytelling is still the core, but we just want to bring it to the internet age. For this first hackathon, we paired filmmakers, playwrights, and screenwriters with creative technologists. We gave them a challenge to create a narrative that spanned multiple platforms and used a specific set of technologies.

GS:  How and why did it get the attention of Forbes, PBS, the Washington Post, and others?

MK: I think it got all the attention it did because it was such a radical idea for traditional filmmakers and writers. We’ve held subsequent hackathons in Los Angeles and across various StoryCode chapters around the world. And the idea has really taken off with other more narrative-driven hackathons like the awesome POV Digital Hackathon series. 

GS: How did you initially get onto the theme of storytelling? Is this a childhood passion you never stopped loving?

MK: I didn’t go to film school. I received a liberal arts education and taught myself how to code. I was a musician in my 20s and ran my own digital agency for 15 years. Towards the end of that run, I realized a pattern that all my passion projects were blending storytelling and code.

Then I saw the interactive video, The Wilderness Downtown, which blew my mind and really set the course for what I’ve done since. That project was so inspiring to me because it was a form in which I was quite familiar, the Music Video. But it was so much more, so much more personal because of the tech they used to create it.

Ever since I’ve been working at Murmur to create our own original projects and to collaborate with filmmakers, adventurous brands, and other storytellers. 

GS: Murmur is in fact the next topic that I’d like to ask you about.  I see that the company creates immersive web sites and cinema. What does “immersive” mean in this context?

MK: When we use the term “immersive” to describe our work, we mean that we are using technology to deeply involve one’s sense in a narrative experience. So instead of a linear film or a traditional web series we create a rich storyworld that the user can navigate and explore. Our project FUTURESTATES is an example of this. It’s an immersive film where different stories are accessed by a 3D model that represents the nodes of the main character’s brain. Sometimes our projects are more focused on extending the story’s experience to another platform.

GS: Can you talk about a client project or two?

MK: Our most recent project is a mobile app version of the Emmy award winning interactive documentary {THE AND}. With THE AND mobile app, you can experience the mechanics and questions from the documentary but with your partner in real life. 

 GS: Could you tell me about Murmur Audio, your indie rock punk audio stock sound collection, which you are building out of sound effects, scores, and more?

MK: I’m always creating music and audio clips, whether for client projects, our original work or just for the fun of it. So I figured that I would just share them with the world. I know how hard it can be to find decent stock audio…so for some folks, hopefully, I can help out here just a bit. But if you don’t like distortion and feedback, then these probably aren’t for you. 

 GS: Murmur has a “mobile audience sentiment app” called “Theater of War.”   What is this about? 

MK: That was a really fun project. We were interested in trying develop a way to gauge how an audience is feeling during a performance. So the app tracks taps during a live dramatic reading. The audience can tap high or low to register the strength of their feeling. We capture that data and then map it to various plot points. This data visualization ends up driving a post-show discussion which can become very interesting. We were initially worried that people wouldn’t tap their phone because they would be too self-conscious, but the opposite happened in actuality. They forgot about their phones and focused on the performance, while still tapping during key moments of resonance. It was fascinating.

GS: What are the skill sets necessary for people who want to make it in your field?

MK: I think the best skill set you can have is an open mind and the confidence to just dive in. Of course you need a basic foundation of design and coding. But we learn best by doing, by making things. So an open mind means you’ll be a good collaborator. Creating these new types of narrative experiences often is a blend of writing, design, software development and filmmaking, so you need to be able to work well with other creative people to succeed. And you need to just “do it” – stop talking and dive in and have the confidence that you will figure it out. A lot of work in this space doesn’t come with a “Hot New Thing”-For-Dummies manual, so you need to work through it yourself and with your collaborators. An open mind and a willingness to put yourself into unfamiliar/uncomfortable situations are key to making great work. 

GS: Who are the ones to watch in creative tech today ? 

MK: I’m a big fan of the work of both VRSE and Specular. VRSE is blazing the trail in VR and Specular is creating amazing open-source photogrammetry software. In my world, which is more story-based, I’m also watching a number of innovative storytellers who use tech, like:

• Topaz Adizes – creator of THE AND.
• Shannon Carroll – creator of Southside Stories. 
• Mark Harris – my partner at Murmur. 
• Ashley Maynor – creator of The Story of The Stuff.
• Kel O’Neill and Eline Jongsma – creators of Empire. 
• Elaine Sheldon – creator of Hollow and the awesome She Does podcast.
• Adnaan Wasey – who runs an amazing series of Story Hackathons at POV Digital.
• Lance Weiler – who’s created too many amazing projects to list here!

GS: What are some challenges that Creative Technologists will grapple with in the coming years?

MK: I think the main challenge is with audience — or more specifically, audience size. If you want to make cool projects for a small, select group of users, cool: go crazy. But if you want to make experiences that can be used by a mass audience, you need to think about platforms. Users are already on platforms (like YouTube, Twitter, Facebook), and it’s much easier to bring an experience to where a user is than to ask them to go to your destination. But by default, a platform limits what the potential experience can be. There are tech limitations to what you can do. So as a creative technologist, you need to decide if you are willing to work within the technology limitations of a platform (where the promise of a bigger audience exists) or if you want to have more freedom and flexibility to do what you want (but it may be harder to attract a large audience to experience it).

GS: What makes New York a great place for creative technology? 
MK: Hands down it’s because NYC is at the crossroads of so many different industries – technology, advertising, film, theater, and publishing. No city has this blend of industries and that makes NYC such a fertile ground for creative technologists. It’s the biggest sandbox and it has the most toys.