LANGSCAPE: Language-driven Architectural VR Scape is an innovative capstone project that aims to develop an immersive and interactive virtual reality (VR) experience, allowing users to control their environment using natural language exclusively. By creating an intuitive and seamless interface between the user and the VR environment, LANGSCAPE enables efficient communication with an AI assistant for designing and manipulating a 3D space. The project's primary goal is to enhance user engagement by bridging the gap between the digital and physical realms by exploring recent advances in Large Language Models (LLM), ultimately providing an innovative and accessible platform for users to explore and create architectural structures.
1. Install Unity Hub.
4. Install Visual Studio Community 2022.
1. Download GIT.
git clone https://github.com/commanser-sar/LANGSCAPE.git
After this, Unity should have populated your directory with all the proper files required to open and execute the project using the Unity Editor. I would recommend sticking with the Visual Studio 2019 that comes with the download, as using VS Code can result in not being able to see the Unity packages with intellisense.
Running the Project requires an active OpenAI Api Key. (Currently used in Assets/Scripts/AI/Command Utility/CommandInterpreter.cs)
->In Windows/Mac/Linux Unity Editor Mode:
- Press and Hold the "v" Key to initiate either LLM
- Include the word "Background" in the recording to access the 2nd LLM
->In Oculus VR - Hand-Tracking:
- Use the Thumbs Up Pose for the Left Hand to initate recording for the 1st LLM - Grid Building
- Use the Scissors Pose for the Left Hand to initiate recording for the 2nd LLM - Object Spawning, Land Changes, Sky Changes, and Related Utility
- Close the Left Hand to end the recording for either LLM
● Editor: Anything placed in this folder will not be compiled at runtime. Place things such as custom inspector menus in here.
● Imported assets: This folder contains all of our imported assets from the unity store that were used for the background system.
● Materials: Contains all Unity materials, as well as our shader graphs.
● Models: Contains the prefabs for the background Objects that are spawned, as well as ground state models, and other 3D models.
● Oculus: Resource folder for Oculus Voice SDK. This might be moved to Oculus Voice.
● Oculus Voice: Oculus Voice SDK package.
● OVR: Contains everything related to VR.
● Prefabs: Contains all the prefabricated GameObjects.
● Rendering Pipeline: Unity package to support our custom shaders.
● Resources: Contains miscellaneous important files that are needed/referenced by the codebase. The LLM prompts are located here.
● Samples: Some packages, such as the OpenAI Utility package, have their example assets and scenes in a separate directory from the rest of the package. This folder contains those example assets.
● Scenes: Contains all LANGSCAPE scenes.
● Scripts: Contains all LANGSCAPE scripts.
● Textures: Contains UI and block textures.
There are some other folders not mentioned here. These are, for the most part, unimportant and can be rearranged without much concern.
git checkout -b <branch_name>
git checkout <existing_branch>
There are two ways to do this, from the command line or GitHub. Using the command line, you have to resolve conflicts manually and could become a hastle. If you make a pull request on GitHub, you get to see what the conflicts are (if any) and pick which files to merge.
git pull origin <branch>
Use this command to verify the branch you are pushing to is correct, and see your changes.
git status
Use this command to add the changes you want to add to the commit.
git add <path/to/filename>
Then, use this command to commit your changes to have them queued for pushing.
git commit -m "<short description of commit>"
Lasty, push your changes into the Github with this command.
git push
Alternatively, if you made changes on the wrong branch, use this command to push.
git push origin <current_branch>:<correct_branch>
If you are not sure how to do it right, or are worried you will break something, try googling it or just ask for help.
To build the project and obtain an executable, you have to specify what kind of build you want. This depends on the operating system and what kind of computer architecture you intend to target.
