🌍 https://www.youtube.com/watch?v=sh46D-C3LU4
🌍 http://lingo-land.herokuapp.com/
Please note the application is currently under reconstruction for performance optimization.
You are welcomed to download our open-source materials and play around.
git clonenpm installnpm start- find the app on http://localhost:8080
Have you ever wanted to learn a foreign language but struggled to simuate an immersive environment? LingoLand is a web based Virtual Reality app that provides a single and multi-user immersive learning experience. Built with A-Frame and Three.js, LingoLand features real time audio communication to interact with other users, speech recognition to interact with bots, and speech synthesis to interact with objects. To ensure a customized user experience, our team used Dialogflow, an Artificial Intelligence API, to train bots to engage in various scenarios and simulate natural language conversations. Come try LingoLand and see for yourself!
Lingo Land uses Virtual Reality to create an immersive language learning experience because immersion is the best way to learn a new language. Starting from the home page, users can choose between environments. Let’s start with Lingo-House, an interactive way to learn situational vocabulary. The user can enter the space, look for clickable items, and learn how to say that word in Spanish. As you can see, users get feedback on their pronunciation. We used web speech API to parse speech to text. The virtual reality environment was created with A-Frame and 3JS. We used Amazon Web Services to store our large 3D models on the cloud.
Another wonderful component of our application is Lingo Café. Lingo Café gives you the opportunity to practice your conversation skills with two different AI bots. You can practice ordering food and drinks by speaking to Jorge our barista bot. You can also practice your conversational skills by meeting someone new and chatting with Juanita, the stranger in the cafe. We built the two AI bots using Dialogflow and web speech APIs. We trained the bots using machine learning so the bots can imitate natural language conversations. The AI bots live on the cloud and were trained for various situations. It makes it so that it doesn’t feel like you’re speaking to the computer. The bots actually listen for what you’re saying and respond accordingly.
At the Lingo Land learning can become a truly international experience. Virtual Lessons can be conducted between multiple users and offer prompts on various topics. To provide peer-to-peer real-time communication we use Networked Aframe and WebRTC API. WebRTC communicates streaming of data between browsers, by leveraging multiple standards and protocols, including WebSockets and JavaScript Session Establishment Protocol.
Our application is designed to help us learn in a new immersive way. We created virtual worlds that engage users in explorational language interactions. We believe if further explored the application can become a basis for peer-to-peer teaching and learning platform.
These fabulous authors of the app are open to new opportunities. Please drop us a line and say hello :]
🌍 Jane Costa @ LinkedIn 🌍 Alla Yakubova @ LinkedIn 🌍 Mahia Mutushy @ LinkedIn 🌍 Alisa Belyakova @ LinkedIn