The virtual pets have been a means of fascination since the late 90’s and have attracted a large number of people who are in many ways uncapable of owning an actual real pet. These virtual pets have ever since their innovation been a friendly companion ruling people’s imaginations, minds and hearts, they behave like a life like companion and confidant of a person and gives them the pleasure of having a friend or a pet that is easy to maintain and spends time with them as per their schedules. Niantic which prominently takes over the gaming market for its AR and VR concepts of gaming, mainly Pokémon GO, has introduced its new, very own virtual pet called the Peridot, which is a brand-new mobile AR pet to play with. This article will delve into the working and development of Peridot and how it captures the users’ attention.
Peridot becomes the first AR game that employs generative AI for the creation of its pets and the virtual world where they live. The game uses the Meta Llama 2 which was integrated by Niantic in order to transform the virtual pets of rather lifelike creatures called the “dots” and converted them to form a virtual interactive creature that inhibits advance behaviors to copy the unpredictable nature of the real-life animals that actually exist around us. Llama 2 is used to give every dot their own unique characteristics which differentiate them from others in the virtual world but make them similar to the ones residing in the real world. To exemplify, just like an owner is not aware of their puppy’s reaction on seeing a cat. Just like their dog would either bark growl or chase it, the dots also react in the same unpredicted way with their surroundings and make their user more and more intrigued. These dots can even interact with real life physical objects and human beings. For example, if you have a pet dog, and you make your dot interact with it, your dot might make the most amusing and unexpected reaction or expression where he might play with your pet happily or roll around with excitement.
The central idea of this innovation exists in the Niantic’s highly advanced ARDK also known as AR Development Kit and the customised Llama 2. The ARDK is employed to convert the real world objects into 3D which further translates them into data or command which is further used by Llama to make them life-like, in-character reactions from the dots.
The use of generative AI in these dots makes them interact with their surroundings in the most adorable way possible where they might cuddle or play with their surroundings or give variety of expressions and impressions. The use Llama 2 helps them demonstrate a vast range of emotions and animations in response to their surrounding world.
Asim Ahmed, who is the global marketing lead for Peridot at Niantic, said in an interview that the use of LLMs like Llama gives them the facility and advantage to completely employ the generative AI in the creation and movement of the Peridot’s in a more realistic and nuanced way so that the characters will appear more like a real pet rather than a virtually created character. He also stated that the Llama 2 also helps the characters or Peridots to become more actively interacted with the real-world environment. The Llama 2 uses its broad range of animations from their wide library so that they would react differently in response to various different situations and objects so as to provide these characters with a more realistic side.
By keeping their users needs in mind, Niantic knew that the people of the modern world are looking for a more realistic and nuanced experience with their dots and how their Peridot has become a major part of their users’ lives. They understood the need of a more engaging factor in their peridots and employed LLMs so that it would learn the basic behaviors of the dots and help them react in certain ways that might create a more interactive and immersive experience for the characters. The Llama LLMs also helps in maintaining the proper animations of advanced quality and rather realistic behaviour with its high programming system which effectively understands the commands and behaviors of the Peridots and help them act in a more sophisticated and adorable way which attracts the user’s attention and captivates them in the ever-evolving virtual world of pets and AR creatures that become their companions in the hectic everyday life. The use of Llama 2 as an open source allows the Peridot team to openly operate lengthy functions such as the development process of the Dots and their playing activities which take up more energy and appear as time consuming.
The more complex transformation of Peridot required a more advanced system that will handle the intense and dynamic interactions of the character and user and provide the user with the deeply engaging experience with their specific dots. The Llama 2 system prioritizes data privacy and iterates and prototypes their virtual environment in order keep the user’s data safe and secure. The open framework of Llama 2 also provides flexibility to the system and empowers the Niantic to increase the development of new and interactive features in the dots and also provides them with the nuances and improved qualities for the generative AI that is used in the system.
The Peridot was initially transformed in November 2023 by Niantic in order to provide a better experience to the users and make Peridot more flexible and life-like and provide the users with a more captivating experience. The Peridot team seemed to explore a better and a rather general purpose LLM in tandem that would go well along with their AR recognition system and will make a better use of their animations library which would make the creatures look more life-like and the work would be cut short as the fine tuning of LLM won’t be required. The implementation of Llama was done in order to reduce the complication of the one-shot prompts and also to put forth the need of enhanced creativity. The risk of Llama being excessively used and exerted was high as these creatures used the unique personality traits that defined and distinguished them from others and also shaped the way in which they would react to their surroundings. Niantic even faced hardships during the creation of prompts that would provide the dots with right amount of expressions and ensuring that the reactions stay intact and consistent with the dots’ movements and change of surroundings.
After the immense popularity and success from the gaming industry and the positive feedback made the Niantic to keep the peridots updating and evolving. The company has ever since worked in the direction to improve and enhance the peridot’s experience and environment and the use of Llama 2 has made the company provide a nuanced experience to the users’ as they can now have conversations with the characters and create a more personalised gaming and AR exposure.
Llama’s most recent upgradation provides the dots to acquire unpredictable and wide range of expressions that does not rely on the particular set of expressions that may seem rather animated and fake instead of real and can be predicted beforehand. The players’ expectations and experiences are heightened with joy and excitement when the dots react with mischief, joy or any other reaction which comes as an organic real-life animation rather than the virtual animations that seem fake and deprive the user with the sense of attachment and interaction.
Niantic employs high quality enhanced computer vision algorithms which transform the physical world images into animated 3D models that looks exactly like those in the real world which allows characters to interact with their surroundings with creativity and intense expressions. By the use of Niantic Lightship ARDK, the dots become capable of understanding their surroundings and physical objects such as flowers, food and pets. The further processing these informations by the custom version of Llama 2, leads to the determination of the dot’s reaction to their new discoveries and manage the dots unique traits such as their personality and history.
The voice recognition or text input allow the users to input any command and interact with their dots. Due to this enhanced feature, there was an enhancement in the latency which was considered in addition to the Llama’s prompt response. The Peridot team defined an expected response format in JSON that helped in the betterment of the LLM response quality which further made the Peridot team to work in complete cooperation with Llama so as to generate a reaction that would make sense of the users’ input and additional information which also included the dots’ need for attention and other features such as their hunger status and other detected objects in the scene.
Asim Ahmed further adds on that they were both surprised an intrigued by the unpredictable behaviour they created by prompting the Llama. He even said that they are looking forward to a consistent and constant upgradation that would help the characters in maintaining and improving their life-like abilities and improve the app into new areas of gaming more procedurally.
How does Peridot make the use of Llama
Understanding of the environment
The AI algorithms of Niantic for computer visions convert the real-life images and visions from the creator’s camera and transform them into what we call a 3D image or a highly advanced AR so that they can freely react and reside in their particular environment.
Passing to the LLM
The words of command are further more transferred to a larger model with advanced language programming system and here the customised Llama 2 comes in use alongside the commands on peridots’ characteristics and personality. The Llama then further decides how the adorable little creature would react to their surroundings based on their personality such as shy, jolly or mischievous.
Final reaction
By the use of Llama, Peridot is able to use the vast library of animations, emotions and reactions that otherwise may seem difficult due to the precise environmental conditions. The use of generative AI makes Dots appear as if they are trying to understand or adapt their surrounding environments or given commands.
By adapting a diversion from the conventional and static algorithms and indulging in the AI-driven spontaneity, the virtual pets of Peridot now provide the experiences which were earlier unpredictable and unexpected.
In conclusion, Niantic’s Peridot transforms the virtual pet world by combining advanced AR technology with the innovative features of Meta’s Llama 2 models. This powerful integration enables Peridot’s virtual creatures, known as “dots,” to exhibit lifelike and unpredictable behaviors, closely resembling real pets. Utilizing generative AI and sophisticated computer vision, these dots interact seamlessly with their environment and users, offering a deeply immersive and engaging experience. Niantic’s AR Development Kit (ARDK) and the customized Llama 2 are pivotal in converting real-world objects into data, enabling lifelike reactions. This technology enhances the emotional depth and realism of the virtual pets, making interactions more spontaneous and authentic. By leveraging Llama 2, Niantic has set a new standard for interactivity in the gaming industry, providing users with a nuanced and dynamic virtual companion.
Continuous updates and improvements to Peridot ensure that the user experience remains captivating and fresh. Enhanced computer vision algorithms and voice recognition features allow the dots to interact creatively and intensely with their surroundings, recognizing objects and responding uniquely. This level of detail and realism not only meets the modern demand for personalized digital experiences but also strengthens the emotional bond between users and their virtual pets. Niantic’s strategic use of Llama 2’s open-source framework facilitates the development of new interactive features while maintaining data privacy, ensuring a secure and flexible user experience. As Peridot evolves, it promises to keep captivating users with its sophisticated and dynamic virtual companions, addressing the growing demand for more authentic and personalized digital interactions. Niantic’s innovative approach heralds a new era in virtual pet companionship, blending technology and creativity to deliver an unparalleled interactive experience.