[DAO:dad9a45] Update #1 for proposal "AI Integration into Decentraland"

Author: 0xedf458ce355c755e1973778b51b917066d4c0bad
Update Status: Late
Project Health: :green_circle: On Track


The project is on track. The report was submitted late due to issues with the website and new rules regarding the fund spending report. It has been reported to GSS.


Grant Update: AI Integration in Decentraland - Progress Report

We’re pleased to share the progress in integrating AI technology into Decentraland, focusing on both backend and frontend solutions to improve virtual interactions with advanced AI capabilities.

Recent Developments:

  1. Backend Library Creation:

    • Developed a backend library compatible with WebSocket libraries like colyseus.js, enabling connections to AI APIs and models such as OpenAI, Claude, and open-source models by ollama.
    • GitHub Repository: LLM Response
    • NPM Package: llm_response on npm
  2. Frontend Toolkit Enhancement:

Test It Out:

  • Experience in Meta Residence Tower: You can now test these developments live in the Meta Residence Tower lobby. Visit Meta Residence Tower in Decentraland to see our AI integration in action.


Next steps

Next Steps:

Moving forward, our focus will be on refining the code, improving documentation, and providing comprehensive examples for easy interaction with our tools. Upcoming milestones include:

  • Polishing the Code: Optimizing our libraries to be efficient and bug-free.
  • Enhancing Documentation: Creating detailed README files that guide users on effectively utilizing our tools.
  • Developing Examples: Offering practical examples on various topics to showcase our AI integration capabilities:
    • Chatbot NPC: How to create NPCs that use data from PDFs, website links, and text files for interaction.
    • Multilingual Q&A Characters: Creating characters that interact in multiple languages for a more inclusive virtual environment.
    • Image Generation: Integrating image generation for immersive experiences.
    • Voice Generation: Adding voice generation for enhanced interactivity.
    • Music Generation: Incorporating music generation to improve ambiance and engagement in virtual spaces.

Additional notes and links

View this update on the Governance dApp