Author: 0xedf458ce355c755e1973778b51b917066d4c0bad
Update Status: Late
Project Health: 
On Track
Introduction
The project is on track. The report was submitted late due to issues with the website and new rules regarding the fund spending report. It has been reported to GSS.
Highlights
Grant Update: AI Integration in Decentraland - Progress Report
We’re pleased to share the progress in integrating AI technology into Decentraland, focusing on both backend and frontend solutions to improve virtual interactions with advanced AI capabilities.
Recent Developments:
-
Backend Library Creation:
- Developed a backend library compatible with WebSocket libraries like colyseus.js, enabling connections to AI APIs and models such as OpenAI, Claude, and open-source models by ollama.
- GitHub Repository: LLM Response
- NPM Package: llm_response on npm
-
Frontend Toolkit Enhancement:
- Forked and enhanced the dcl-npc-toolkit to support backend communication and UI interactions, specifically designed for AI applications in Decentraland.
- GitHub Repository: DCL NPC Toolkit - AI Version
- NPM Package: dcl-npc-toolkit-ai-version on npm
Test It Out:
- Experience in Meta Residence Tower: You can now test these developments live in the Meta Residence Tower lobby. Visit Meta Residence Tower in Decentraland to see our AI integration in action.
Blockers
Next steps
Next Steps:
Moving forward, our focus will be on refining the code, improving documentation, and providing comprehensive examples for easy interaction with our tools. Upcoming milestones include:
- Polishing the Code: Optimizing our libraries to be efficient and bug-free.
- Enhancing Documentation: Creating detailed README files that guide users on effectively utilizing our tools.
- Developing Examples: Offering practical examples on various topics to showcase our AI integration capabilities:
- Chatbot NPC: How to create NPCs that use data from PDFs, website links, and text files for interaction.
- Multilingual Q&A Characters: Creating characters that interact in multiple languages for a more inclusive virtual environment.
- Image Generation: Integrating image generation for immersive experiences.
- Voice Generation: Adding voice generation for enhanced interactivity.
- Music Generation: Incorporating music generation to improve ambiance and engagement in virtual spaces.