Dynamically Constructing a Google Assistant Carousel

Note: if you are purely here for guidance on the technical problem indicated in the title, jump down to “The Solution.”

By day, I work at a family-owned business that provides software and services to libraries. A while back, I got interested in creating a proof-of-concept integration between our software and a voice assistant. I’ve recently had the bandwidth to scratch that itch. I decided to start with Google Assistant, figuring that owning a Pixelbook and Pixel 2 phone would give me some advantages.

Although the juicier ambition is to assist patrons and students with checking due dates, renewing loans and other more transaction-based use cases, I started with something simple: search. Although the majority of our software is built on APIs, they are still private for lack of the management tools necessary to support developers, such as version control, deprecation, docs, and test consoles (something I’m working on fixing, but that’s a different post). Search, however, I could work with pretty easily. And, when learning a new framework and design space, experience has taught me not to be too ambitious from the start.

Here was the goal I started with:

• User asks to search the library catalog.
• The assistant uses key phrases such as “by” or “about” to choose from the following intents: title, author, subject, or series.
• The assistant performs the search and indicates the number of results. For now, the assistant is limited to one page of up to 10 results.
• For voice-only interfaces, the user can indicate a result number to get more detail on the title record.
• For screen interfaces, the result set will be rendered as a carousel, including covert art for each title, if available.

The Problem

For this to work, I needed to dynamically construct a carousel using non-deterministic elements. However, all of the tutorials and code examples provided by Google show only carousels based on a fixed number of already-known elements. Even worse, after a significant amount of time searching Google, reading StackOverflow articles and digging through the API documentation, I could not find a working example. All the examples I found either used a deprecated API or were presumably speculative approaches that people had suggested in good faith, but clearly not tested themselves.

Here are the two general approaches (with several variants) that I tried along the way:

• Create an empty carousel, construct each item in a loop and then add each item to the carousel. You cannot create an empty carousel; it must be constructed with the set of items already prepared.
• Create a carousel with one item, then immediately remove that item and add additional items. You cannot remove items from a carousel once it has been created, even though you can add items.

In case you are unclear about the distinction between deterministic and non-deterministic item sets, consider these two use cases:

1 — Build a carousel for menu items at your restaurant

Presumably, at any given time your restaurant has a set menu. Since you know the menu in advance, you could simply build your carousel to reflect the current menu items. When the menu changed, you could update your carousel. While this would be easier to maintain if it were dynamic, it’s an acceptable solution.

Incidentally, in the Google tutorial, you build a carousel to let someone choose a favorite color. They are only given three options, and the same three options every time. Luckily, two of them were blue, so I didn’t feel left out!

2 — Build a carousel for the results of a keyword search

Since you have no idea in advance what the user will search for, you don’t know how many items will be displayed in the carousel, nor do you know any information about the items. Therefore, the carousel would have to be constructed anew in code each time a search is performed.

The Solution

Ultimately, what worked was to create an empty items object, construct each item as another object in a loop, add them to the items object and then finally build the carousel off of the items object.

Now, a more seasoned javascript developer might have figured this out faster. I suffered from two handicaps. First, I’ve been doing Python and C# for ages (games), so I’m extremely rusty at javascript, and it has changed substantially since last I did much with it. And second, I tried to follow the API documentation which referenced OptionItem and OptionInfo classes as the elements of a carousel.

So, I tried for a while to import those classes, build them with constructors and then stuff them in the carousel. As it turns out, they have no constructors — you cannot actually build them directly, but only provide generic javascript objects in the right format to get converted by the carousel constructor. Oi.

Here’s what I ended up with:

function searchResultCarousel(results) {
  var items = {};
results.forEach((result, index) => {
    console.log(`Building item from result: ${result.title}`);
items[index] = {
      optionInfo: {
        key: (index + 1).toString(),
        synonyms: result.title,
      },
      title: result.title,
      url: ``,
      image: new Image({
        url: `[REDACTED DUE TO TRADE SECRETS]`,
        alt: `Cover art for: ${result.title} - ${result.year}`
      }),
    }
  })
return new Carousel({
    title: 'Search Results',
    items: items
  });
};

where the results object is a JSON object returned from the search API.

And here’s what it looks like in action:

Simulated voice search for subject = Aldo Leopold, showing search result carousel with cover images.

If I could recommend one improvement to the Actions on Google API documentation, it would be this:

Show examples of dynamically constructed lists and carousels. It is both not very intuitive and there are a lot of people trying to figure it out, based on the many dead-end forum posts I found.

Also, credit to this chap for having the solution, but also helping me fix a crucial problem with []s that should have been {}s. I had, by that point, actually figured out the right assembly approach to the carousel items, but his loop was a lot more elegant, so I learned from that too.

So, that’s it. If you are trying to accomplish the same goal and running into similar hurdles, I hope this write-up moves you closer to a working solution. If you have any questions or feedback, hit me up here, Twitter, wherever.