[New Intelligence Guide] The popular Stanford Town has been open-sourced for a week, and someone has already started creating it. The first AI simulated town, AI Town, was born, with 8 intelligent agents living and socializing in the “Westworld”.
The Stanford AI agent town creation is here!
Just now, a team built a real “Westworld” AI Town based on the Stanford Smallville project.
And everyone can try it online.
Project address: https://www.convex.dev/ai-town
A few days ago, Stanford officially open-sourced the popular 25 intelligent agent town, garnering 9.9k stars on GitHub.
Unexpectedly, within a week, a well-known venture capital company personally stepped in,reproducing a vivid Westworld!
As Nvidia senior research scientist Jim Fan said:
When a venture capital company is eager to form a team to replicate an AI paper and open up a platform, you know this idea has reached peak popularity!
Soon, I can imagine the entire world, including pixel art and maps, being generated by AI. New characters will be automatically generated, and even the physical rules within the game may be rewritten instantly. Never underestimate the creativity of the entire open-source software community.
What are the 8 intelligent agents talking about?
Next, let’s introduce who the residents of this town are and what they talked about today.
The young man in a black suit with golden hair is Alex.
He enjoys painting, programming, and reading science fiction books. He is kind, but he also expresses his views in a teasing manner. One more thing, he particularly dislikes answering the same question repeatedly.
Alice is an outstanding scientist who, with her intelligence and insight, discovers mysteries in the universe that no one can solve.
When she explains her discoveries, she often uses abstract language. She gives the impression of being incomprehensible and forgetful.
Peter is a devout believer who tends to explain everything from a religious perspective.
When talking to others, he often expresses his passion for faith or warns others to follow the right path and avoid sin.
The old man with gray hair is Bob.
His personality is somewhat reclusive, so he enjoys gardening, which is a solitary activity. When others talk to him, he will respond but is not willing to engage in long conversations. Deep down, he is somewhat self-conscious about not having attended college.
The little black figure is Kurt. He is haunted by something that happened in the past, making it difficult for him to speak about it, and it often lingers in his mind. He worries that others will notice him deliberately avoiding this topic, which torments him.
Although this experience has brought some trouble to his life, he has been working hard to slowly step out of the shadow and start anew.
Across from Kurt is Kira. She appears cheerful and lively on the surface, often discussing happy topics to mask her inner pain. However, when she is alone, she often cannot suppress her sadness and ends up in tears.
She has been working hard to deal with these negative emotions, but sometimes it’s still difficult to fully let go, and she seems troubled.
The girl with pink hair is Stella. She is untrustworthy and always finds ways to deceive others, usually for money or for her own profit. She possesses extraordinary charm and does not hesitate to use this advantage to achieve her goals.
The young man in a blue top with silver-gray hair is Lucky.
His personality is as his name suggests, cheerful and lively, full of curiosity about the world. He enjoys reading science history and dreams of exploring the galaxy in a spaceship. He is good at communication and is friendly.
As above, these are the character settings of all the characters in AI Town, which really resembles the Westworld amusement park.
So, what are these 8 residents talking about? How do they live and socialize?
Lucky has just completed a space exploration journey. He talks about his experience with everyone he meets. The irritable Bob is still only interested in trimming plants.
– Hey, Bob! Long time no see. How have you been recently?
– As always, just busy with gardening.
– I see! I recently participated in a magical space adventure! I explored a distant planet; it was incredible.
– That sounds great, but I’m not very interested. I need to get back to planting flowers.
– No problem, Bob. If you’re interested, we can chat about this adventure anytime.
– Goodbye, my friend. I’m going to get busy with trimming.
– Hey, Alex! We haven’t seen each other for a long time. How have you been recently?
– Hey, Kira! I’m good, thanks for asking. I’m still painting and programming. How about you?
– I’m glad to hear that. I’ve been busy practicing yoga and exploring new recipes. It helps me forget my worries. Have you had any exciting adventures lately?
– Not really any exciting adventures. Just busy with my current painting and programming projects. But finding inner peace through books is a good way; do you have any particular genres you’re interested in?
– Well, books have always been a great escape for me. I love immersing myself in the worlds of science fiction and fantasy, where everything is vastly different from reality. It’s like a temporary breath of fresh air.
– Science fiction is also my favorite. Have you read “Dune”? It’s a breathtaking sci-fi epic that might take you into a whole new universe.
As we can see, everyone’s conversation content is most related to their personal character preferences. The character settings have determined how they live and even how they socialize.
One downside is that while the dialogue content of these people may seem somewhat clichéd, it sounds real enough.
AI Town, accessible to everyone
AI Town is a virtual town where AI characters live, chat, and socialize.
This project is a deployable JavaScript starter kit for easily building and customizing your own AI Town.
Project address: https://github.com/a16z-infra/ai-town
The main goal of this project is not only to make work fun but also to provide a powerful and scalable platform.
The backend engine natively supports shared global state, multi-agent transactions, and logs of all events, making it suitable for projects ranging from simple ones to multiplayer games.
The second goal is to provide a usable JS/TS framework. Most simulators in this field (including Stanford’s original paper) are written in Python.
Complete Technology Stack
– Game engine and database (Game engine & Database): Convex
– Vector database (VectorDB): Pinecone
– Authentication (Auth): Clerk
– Text generation model (Text model): OpenAI
– Deployment (Deployment): Fly
– Pixel art generation (Pixel Art Generation): Replicate, Fal.ai
Notable netizen “Baoyu” in the industry believes that compared to the original implementation, the new tech stack rewritten with Nextjs and Tailwind is more friendly for the frontend.
Netizens getting creative
In fact, some netizens have already built their own “Cat Town” based on the newly open-sourced AI Town.
Additionally, some netizens plan to create a branded intelligent agent town.
Some netizens have expressed that AI-generated agents are a new trend, and even venture capital companies have recognized this.
This open-source library allows anyone to create a “digital Westworld” where AI agents live and communicate with each other, unaware that they live in a simulated environment.
This marks the beginning of a new era of AI-generated content, not limited to games and entertainment.
Imagine, you could simulate how people interact in a workspace, simulate digital asset custody in banks, or even simulate operations in satellite command and control centers.
Installation
Clone the repository and install the packages
git clone [email protected]:a16z-infra/ai-town.gitcd AI-townnpm installnpm run dev
– Visit https://dashboard.clerk.com/ and click “Add Application” to add an application
– Name the application and choose the login method provided to users
– Add NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY and CLERK_SECRET_KEY to the .env.local file
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_***
CLERK_SECRET_KEY=sk_***
– Go to the JWT templates and create a new Convex template
– Copy the URL of the JWKS endpoint for use in the steps below
To use OpenAI’s language model, visit https://platform.openai.com/account/api-keys to get your OpenAI API key.
– Visit https://app.pinecone.io/ and create a Pinecone index, then click “Create Index”
– Name the index (this will be the environment variable PINECONE_INDEX_NAME)
– Set the Dimension to 1536
– After creating the index, click “API Keys” in the left navigation bar and create an API key: copy the “Environment” value to the PINECONE_ENVIRONMENT variable, and the “Value” to the PINECONE_API_KEY
d. Add secrets in the Convex dashboard
npx convex dashboard
Go to “Settings” and add the following environment variables. Here, CLERK_ISSUER_URL is the URL of the JWKS endpoint (from step a).
OPENAI_API_KEY sk-*
CLERK_ISSUER_URL https://*
PINECONE_API_KEY **
PINECONE_ENVIRONMENT us*
PINECONE_INDEX_NAME **
Run both the frontend and backend simultaneously:
npm run dev
Now you can access http://localhost:[PORT_NUMBER].
If you want to run the frontend in another terminal outside of Convex (the backend functions will sync on save), you can run the following two commands:
npm run dev:frontend
npm run dev:backend
Note: If the window is idle, the simulation will pause after 5 minutes.
Run/Test/Debate
Add a new world, seed it, and start running:
npx convex run init:reset
Iterate sequentially and create a world:
npx convex run --no-push init:resetFrozen
# for each iteration
npx convex run --no-push engine:tick '{"worldId":"<your world id>","noSchedule":true}'
If there is too much activity, freeze the backend:
npx convex run --no-push engine:freezeAll
# when ready to rerun (defaults to latest world)
npx convex run --no-push engine:unfreeze
There are many ways to clear all databases:
– Go to the dashboard npx convex dashboard and clear the tables from there
– Adjust variables in crons.ts to automatically clean up old logs and memory entries
– Run npx convex run –no-push testing:debugClearAll to clear all tables
– If you encounter issues, check the origin/reset-town git branch. Running npm run dev there will clear the mode, stop functions, and allow you to delete tables in the dashboard
To remove all vectors from the Pinecone index, you can run:
npx convex run --no-push lib/pinecone:deleteAllVectors
Note: If the index is shared between development and production or between projects, they will all be cleared. Typically, there is no need to delete vectors from Pinecone, as each query is indexed by userId, which is unique.
In the side terminal, run the following command to listen to messages:
npx convex run testing:listMessages --no-push --watch
Or observe the state of a character:
npx convex run testing:latestPlayer --no-push --watch
Additionally, more functions can be found in testing.ts.
Deploy the Application
– Register an account on fly.io and install flyctl
– If using Github Codespaces, you need to install flyctl and authenticate from the codespaces cli by running fly auth login
– Run npx convex deploy to deploy the development environment to production. Make sure to copy all secret information to the production environment of Convex
– Run fly launch in the root directory of the project, which will generate a fly.toml file containing all configurations
– Modify the generated fly.toml file and add NEXT_PUBLIC_* for NextJS during the build so that it can be accessed on the client side
[build]
[build.args]
NEXT_PUBLIC_CLERK_SIGN_IN_URL="/sign-in"
NEXT_PUBLIC_CLERK_SIGN_UP_URL="/sign-up"
NEXT_PUBLIC_CLERK_AFTER_SIGN_IN_URL="/"
NEXT_PUBLIC_CLERK_AFTER_SIGN_UP_URL="/"
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY="pk_*****"
NEXT_PUBLIC_CONVEX_URL="https://*******.convex.cloud"
– Modify the Dockerfile generated by fly.io, adding new environment variables before RUN npm run build
ARG NEXT_PUBLIC_CLERK_SIGN_IN_URL
ARG NEXT_PUBLIC_CLERK_SIGN_UP_URL
ARG NEXT_PUBLIC_CLERK_AFTER_SIGN_IN_URL
ARG NEXT_PUBLIC_CLERK_AFTER_SIGN_UP_URL
ARG NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY
ARG NEXT_PUBLIC_CONVEX_URL
# Build application
RUN npm run build
– Run fly deploy –ha=false to deploy the application. Use the –ha flag to ensure that fly only starts one instance, which is provided in the free plan
– Run fly scale memory 512 to scale the memory of the fly virtual machine for this application.
– Create a new .env.prod file locally and fill in all production environment secret information. Remember to update NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY and CLERK_SECRET_KEY by copying the secret information from Clerk’s production instance, and upload it using -cat .env.prod | fly secrets import. Also, update CONVEX_DEPLOYMENT and NEXT_PUBLIC_CONVEX_URL
Customize Your Own Simulation
Note: Since character data will be sent to Convex during the initial load, you should rerun npx convex run testing:debugClearAll –no-push after each change, and then run npm run dev to re-upload everything to Convex. However, npx convex run testing:debugClearAll –no-push will clear all data, including vector storage.
1. Create your own characters and stories: All character and story data, along with their sprite references, are stored in data.ts.
2. Update the sprites: In data.ts, you will see the following code:
{ name: 'f1', textureUrl: '/assets/32x32folk.png', spritesheetData: f1SpritesheetData, speed: 0.1, },...
You should find a sprite for the character and define the sprite actions/assets in the corresponding file (in the above example, f1SpritesheetData is defined in f1.ts).
3. Update the background (environment): convex/maps/firstmap.ts is where the map is loaded. The easiest way to export a map is to use Tiled — Tiled will export it as CSV, which can then be converted into a 2D array that firstmap.ts can accept.
Simulation theory suggests that the world we live in is essentially a simulation created by a more advanced civilization, like a computer simulation.
That is to say, as civilizations progress, they can create their own realistic simulations, making it less likely that the world we live in is the base reality.
While this is somewhat difficult to comprehend, the latest developments in AI Town bring this theory closer to reality and may mark a key moment in the history of AI development.
If AI can create a realistic world where characters can interact, there will be various applications, such as replacing NPCs in games.
At first glance, it is difficult to determine whether these characters remember the conversations they have had and use those conversations as input for future dialogues.
But if that is the case, we may have the first AI simulated town.
https://github.com/a16z-infra/ai-town#customize-your-own-simulation
