Artificial Intelligence

Award-winning innovators Caroline Lair and Lucia Gallardo will be speaking at TNW Conference, which takes place on June 15 & 16 in Amsterdam. If you want to experience the event (and say hi to our editorial team!), we’ve got something special for our loyal readers. Use the promo code READ-TNW-25 and get a 25% discount on your business pass for TNW Conference. See you in Amsterdam!

Social inequality and climate risk have become central to understanding what will drive innovation – and investment – for the future. On day two of TNW Conference, Caroline Lair, founder of startup and scaleup communities The Good AI and Women in AI, and Lucia Gallardo, founder and CEO of impact innovation “socio-technological experimentation lab” Emerge, will be on the Growth Quarters stage for a session titled “Technology-Driven Climate Justice.” 

The climate crisis is itself the result of a deeply embedded and systemic exploitation of nature and people in the name of profit. Its impact is already being felt disproportionately over the world, with severe heat waves, droughts, and entire nations disappearing below sea level. What’s more, the people worst affected are those who have contributed little to the greenhouse gas emissions driving global warming.

Climate justice is the idea that climate change is not just an environmental but also a social justice issue, and aims to ensure that the transition to a low-carbon economy is equitable and benefits everyone. Lair and Gallardo will specifically speak about how technologies such as AI, blockchain, and Web3 can play a crucial role in addressing these issues. 

AI for good

Artificial intelligence can be applied in the quest for climate justice in several ways, given that it is implemented in a way that ensures transparency, accountability, and fairness. These include data analysis and prediction, discovering patterns and informing policies, as well as evaluating their effectiveness. 

It can also enhance climate modelling capabilities, crucial for developing adaptation strategies. Furthermore, AI-powered technologies can monitor, for instance, weather systems with real-time data and also optimise resource allocation and energy distribution.

Reimagining value

Emerge’s objective is to “reimagine impact innovation with regenerative monetisation models.” Regenerative finance goes beyond traditional models that focus on profit, taking into account broader social, environmental, and economic impacts. 

Blockchain technology can, for instance, offer transparency for transactions, ensuring that funds are indeed directed to regenerative investments. It can also tokenise regenerative assets such as renewable energy installations, sustainable agriculture initiatives, or ecosystem restoration projects, representing them as digital tokens and making them more accessible to a broader range of investors. 

Meanwhile, in the words of Gallardo, “Integrating crypto into existing ecological initiatives doesn’t automatically mean it is applied regenerative finance. We must be intentional about how we’re reimagining value.”

Reclaiming an equitable future

Why am I looking forward to this session? The theme of this year’s TNW Conference is “Reclaim The Future”. In all honesty, I belong to a generation that, while I hopefully have several decades more of on-earth experience ahead of me, will most likely not have to deal with full-on dystopian scenarios, battling to survive climate catastrophe.

I am also privileged in terms of geographical location and socioeconomic status not to have to worry about immediate drought and famine. (Flooding may be another matter, but as someone said when convincing me to move to Amsterdam – “wouldn’t you prefer to live in a place that is already used to keeping water out?”) 

However, this does not mean that we who enjoy such privileges get to simply shrug our shoulders and carry on indulging in business as usual. TNW has always been about the good technology can do in the world. And what is better than employing it in service of one of the greatest challenges of our time?

The latest version of the project introduces Skyrim scripting for the first time, which the developer says allows for lip syncing of voices and NPC awareness of in-game events. While still a little rigid, it feels like a pretty big step towards climbing out of the uncanny valley.

Here’s how ‘Art from the Machine’ describes the project in a recent Reddit post showcasing their work:

A few weeks ago I posted a video demonstrating a Python script I am working on which lets you talk to NPCs in Skyrim via ChatGPT and xVASynth. Since then I have been working to integrate this Python script with Skyrim’s own modding tools and I have reached a few exciting milestones:

NPCs are now aware of their current location and time of day. This opens up lots of possibilities for ChatGPT to react to the game world dynamically instead of waiting to be given context by the player. As an example, I no longer have issues with shopkeepers trying to barter with me in the Bannered Mare after work hours. NPCs are also aware of the items picked up by the player during conversation. This means that if you loot a chest, harvest an animal pelt, or pick a flower, NPCs will be able to comment on these actions.

NPCs are now lip synced with xVASynth. This is obviously much more natural than the floaty proof-of-concept voices I had before. I have also made some quality of life improvements such as getting response times down to ~15 seconds and adding a spell to start conversations.

When everything is in place, it is an incredibly surreal experience to be able to sit down and talk to these characters in VR. Nothing takes me out of the experience more than hearing the same repeated voice lines, and with this no two responses are ever the same. There is still a lot of work to go, but even in its current state I couldn’t go back to playing without this.

You might notice the actual voice prompting the NPCs is also fairly robotic too, although ‘Art from the Machine’ says they’re using speech-to-text to talk to the ChatGPT 3.5-driven system. The voice heard in the video is generated from xVASynth, and then plugged in during video editing to replace what they call their “radio-unfriendly voice.”

And when can you download and play for yourself? Well, the developer says publishing their project is still a bit of a sticky issue.

“I haven’t really thought about how to publish this, so I think I’ll have to dig into other ChatGPT projects to see how others have tackled the API key issue. I am hoping that it’s possible to alternatively connect to a locally-run LLM model for anyone who isn’t keen on paying the API fees.”

Serving up more natural NPC responses is also an area that needs to be addressed, the developer says.

For now I have it set up so that NPCs say “let me think” to indicate that I have been heard and the response is in the process of being generated, but you’re right this can be expanded to choose from a few different filler lines instead of repeating the same one every time.

And while the video is noticeably sped up after prompts, this mostly comes down to the voice generation software xVASynth, which admittedly slows the response pipeline down since it’s being run locally. ChatGPT itself doesn’t affect performance, the developer says.

This isn’t the first project we’ve seen using chatbots to enrich user interactions. Lee Vermeulen, a long-time VR pioneer and developer behind Modbox, released a video in 2021 showing off one of his first tests using OpenAI GPT 3 and voice acting software Replica. In Vermeulen’s video, he talks about how he set parameters for each NPC, giving them the body of knowledge they should have, all of which guides the sort of responses they’ll give.

Check out Vermeulen’s video below, the very same that inspired ‘Art from the Machine’ to start working on the Skyrim VR mod:

As you’d imagine, this is really only the tip of the iceberg for AI-driven NPC interactions. Being able to naturally talk to NPCs, even if a little stuttery and not exactly at human-level, may be preferable over having to wade through a ton of 2D text menus, or go through slow and ungainly tutorials. It also offers up the chance to bond more with your trusty AI companion, like Skyrim’s Lydia or Fallout 4’s Nick Valentine, who instead of offering up canned dialogue might actually, you know, help you out every once in a while.

And that’s really only the surface level stuff that a mod like ‘Art from the Machine’ might deliver to existing games that aren’t built with AI-driven NPCs. Imagining a game that is actually predicated on your ability to ask the right questions and do your own detective work—well, that’s a role-playing game we’ve never experienced before, either in VR our otherwise.