Bringing Real-time Spatial Audio to the Web with Songbird

For a virtual scene to be truly immersive, stunning visuals need to be accompanied by true spatial audio to create a realistic and believable experience. Spatial audio tools allow developers to include sounds that can come from any direction, and that are associated in 3D space with audio sources, thus completely enveloping the user in 360-degree sound.

Spatial audio helps draw the user into a scene and creates the illusion of entering an entirely new world. To make this possible, the Chrome Media team has created Songbird, an open source, spatial audio encoding engine that works in any web browser by using the Web Audio API.

The Songbird library takes in any number of mono audio streams and allows developers to programmatically place them in 3D space around the user. Songbird allows you to create immersive soundscapes, realistically reproducing reflection and reverb for the space you describe. Sounds bounce off walls and reflect off materials just as they would in real-life, capturing truly 360-degree sound. Songbird creates an ambisonic soundfield that can then be rendered in real-time for use in your application. We’ve partnered with the Omnitone project, which we blogged about last year, to add higher-order ambisonic support to Omnitone’s binaural renderer to produce far more accurate sounding audio than ever before.

Songbird encapsulates Omnitone and with it, developers can now add interactive, full-sphere audio to any web based application. Songbird can scale to any order ambisonics, thereby creating a more realistic sound and higher performance than what is achievable through standard Web Audio API.

Songbird Audio Processing Diagram

The implementation of Songbird is based on the Google spatial media specification. It expects mono input and outputs ambisonic (multichannel) ACN channel layout with SN3D normalization. Detailed documentation may be found here.

As the web emerges as an important VR platform for delivering content, spatial audio will play a vital role in users’ embrace of this new medium. Songbird and Omnitone are key tools in enabling spatial audio on the web platform and establishing it as a preeminent platform for compelling VR experiences. Combining these audio experiences with 3D JavaScript libraries like three.js gives a glimpse into the future on the web.

Demo combining spatial sound in 3D environment

This project was made possible through close collaboration with Google’s Daydream and Web Audio teams. This collaboration allowed us to deliver similar audio capabilities to the web as are available to developers creating Daydream applications.

We look forward to seeing what people do with Songbird now that it's open source. Check out the code on GitHub and let us know what you think. Also available are a number of demos on creating full spherical audio with Songbird.

By Jamieson Brettle and Drew Allen, Chrome Media Team

Authenticating to HashiCorp Vault using Google Cloud IAM

Applications often require access to small pieces of sensitive data at build or run time, referred to as secrets. Secrets are generally more sensitive than other environment variables or parts of your repository as they may grant access to additional data, such as user data.

HashiCorp Vault is a popular open source tool for secret management, which allows a developer to store, manage and control access to tokens, passwords, certificates, API keys and other secrets. Vault has many options for authentication, called authentication backends. These allow developers to use many kinds of identities to access Vault, including tokens, or usernames and passwords. As the number of developers on a team grows, these kinds of authentication options become impractical; and in enterprise scenarios, managing and auditing these identities becomes burdensome.

Today, we are pleased to announce a Google Cloud Platform IAM authentication backend for Vault. This allows a developer to use an existing IAM identity to authenticate to Vault. Using a service account, you can sign a JWT to show it came from a particular account, and use that to authenticate to Vault. Learn more in the documentation.

The following example in Go shows how a user can authenticate with Vault using this backend. This example assumes the Vault server has already been mounted at auth/gcp and configured.

package main

import (
 ...
 vaultapi "github.com/hashicorp/vault/api"
 "golang.org/x/oauth2"
 "golang.org/x/oauth2/google"
 "google.golang.org/api/iam/v1"
 ...
)

func main() {
 // Start [PARAMS]
 project := "project-123456"
 serviceAccount := "myserviceaccount@project-123456.iam.gserviceaccount.com"
 credsPath := "path/to/creds.json"

 os.Setenv("VAULT_ADDR", "https://vault.mycompany.com")
 defer os.Setenv("VAULT_ADDR", "")
 // End [PARAMS]

 // Start [GCP IAM Setup]
 jsonBytes, err := ioutil.ReadFile(credsPath)
 if err != nil {
  log.Fatal(err)
 }
 config, err := google.JWTConfigFromJSON(jsonBytes, iam.CloudPlatformScope)
 if err != nil {
  log.Fatal(err)
 }

 httpClient := config.Client(oauth2.NoContext)
 iamClient, err := iam.New(httpClient)
 if err != nil {
  log.Fatal(err)
 }
 // End [GCP IAM Setup]
 
 // 1. Generate signed JWT using IAM.
 resourceName := fmt.Sprintf("projects/%s/serviceAccounts/%s", project, serviceAccount)
 jwtPayload := map[string]interface{}{
  "aud": "auth/gcp/login",
  "sub": serviceAccount,
  "exp": time.Now().Add(time.Minute * 10).Unix(),
 }

 payloadBytes, err := json.Marshal(jwtPayload)
 if err != nil {
  log.Fatal(err)
 }
 signJwtReq := &iam.SignJwtRequest{
  Payload: string(payloadBytes),
 }

 resp, err := iamClient.Projects.ServiceAccounts.SignJwt(
resourceName, signJwtReq).Do()
 if err != nil {
  log.Fatal(err)
 }

 // 2. Send signed JWT in login request to Vault.
 vaultClient, err := vaultapi.NewClient(vaultapi.DefaultConfig())
 if err != nil {
  log.Fatal(err)
 }

 vaultResp, err := vaultClient.Logical().Write(
"auth/gcp/login", 
map[string]interface{}{
   "role": "test",
   "jwt":  resp.SignedJwt,
  })

 if err != nil {
  log.Fatal(err)
 }

 // 3. Use auth token from response.
 log.Println("Access token %s", vaultResp.Auth.ClientToken)
 vaultClient.SetToken(vaultResp.Auth.ClientToken)
 // ...
}

Vault is just one way of managing secrets in development. For further reading on choosing a solution that’s right for you, see Google Cloud Platform’s documentation on Secret Management.

By Emily Ye, Software Engineer

Making Great Mobile Games with Firebase

So much goes into building and maintaining a mobile game. Let’s say you want to ship it with a level builder for sharing content with other players and, looking forward, you want to roll out new content and unlockables linked with player behavior. Of course, you also need players to be able to easily sign into your soon-to-be hit game.

With a DIY approach, you’d be faced with having to build user management, data storage, server side logic, and more. This will take a lot of your time, and importantly, it would take critical resources away from what you really want to do: build that amazing new mobile game!

Our Firebase SDKs for Unity and C++ provide you with the tools you need to add these features and more to your game with ease. Plus, to help you better understand how Firebase can help you build your next chart-topper, we’ve built a sample game in Unity and open sourced it: MechaHamster. Check it out on Google Play or download the project from GitHub to see how easy it is to integrate Firebase into your game.

Before you dive into the code for Mecha Hamster, here’s a rundown of the Firebase products that can help your game be successful.

Analytics

One of the best tools you have to maintain a high-performing game is your analytics. With Google Analytics for Firebase, you can see where your players might be struggling and make adjustments as needed. Analytics also integrates with Adwords and other major ad networks to maximize your campaign performance. If you monetize your game using AdMob, you can link your two accounts and see the lifetime value (LTV) of your players, from in-game purchases and AdMob, right from your Analytics console. And with Streamview, you can see how players are interacting with your game in realtime.

Test Lab for Android - Game Loop Test

Before releasing updates to your game, you’ll want to make sure it works correctly. However, manual testing can be time consuming when faced with a large variety of target devices. To help solve this, we recently launched Firebase Test Lab for Android Game Loop Test at Google I/O. If you add a demo mode to your game, Test Lab will automatically verify your game is working on a wide range of devices. You can read more in our deep dive blog post here.

Authentication

Another thing you’ll want to be sure to take care of before launch is easy sign-in, so your users can start playing as quickly as possible. Firebase Authentication can help by handling all sign-in and authentication, from simple email + password logins to support for common identity providers like Google, Facebook, Twitter, and Github. Just announced recently at I/O, Firebase also now supports phone number authentication. And Firebase Authentication shares state cross-device, so your users can pick up where they left off, no matter what platforms they’re using.

Remote Config

As more players start using your game, you realize that there are few spots that are frustrating for your audience. You may even see churn rates start to rise, so you decide that you need to push some adjustments. With Firebase Remote Config, you can change values in the console and push them out to players. Some players having trouble navigating levels? You can adjust the difficulty and update remotely. Remote Config can even benefit your development cycle; team members can tweak and test parameters without having to make new builds.

Realtime Database

Now that you have a robust player community, you’re probably starting to see a bunch of great player-built levels. With Firebase Realtime Database, you can store player data and sync it in real-time, meaning that the level builder you’ve built can store and share data easily with other players. You don't need your own server and it’s optimized for offline use. Plus, Realtime Database integrates with Firebase Auth for secure access to user specific data.

Cloud Messaging & Dynamic Links

A few months go by and your game is thriving, with high engagement and an active community. You’re ready to release your next wave of new content, but how can you efficiently get the word out to your users? Firebase Cloud Messaging lets you target messages to player segments, without any coding required. And Firebase Dynamic Links allow your users to share this new content — or an invitation to your game — with other players. Dynamic Links survive the app install process, so a new player can install your app and then dive right into the piece of content that was shared with him or her.

At Firebase, our mission is to help mobile developers build better apps and grow successful businesses. When it comes to games, that means taking care of the boring stuff, so you can focus on what matters — making a great game. Our mobile SDKs for C++ and Unity are available now at firebase.google.com/games.

By Darin Hilton, Art Director