Let’s face it. You didn’t click on this article by accident. You’re as much of a Pokémon nerd as I am and there’s complete reason to feel excited given what I’m about to show you. A YouTuber by the name of Abe’s Projects decided to throw together a few components to make a rudimentary (but functioning) Pokédex and I CANNOT KEEP CALM!
This Pokédex works surprisingly like the original. Relying on the powers of ChatGPT to identify imagery captured through a rather basic camera setup, Abe’s Pokédex does a fairly good job of replicating the experience of the original from the hit TV series and comic book. Abe even encased his electronics in a wonderfully nostalgic red 3D-printed enclosure, making it resemble the original Pokédex to an uncanny degree… and if that wasn’t enough, he even programmed the Pokédex to speak just like the original, with a computer-ish robotic voice.
Designer: Abe’s Projects
The process, although fairly complicated, gets detailed out by Abe in the YouTube video. One of his admittedly harder builds, Abe mentions the first conundrum – planning the exterior and interior. The problem – you can’t 3D model an outer shape without knowing where your inner components are going to sit, and you can’t know where your inner components are going to sit without planning out your outer shell. Nevertheless, Abe designed a rudimentary framework featuring an outer shell, a few removable components (like the bezel for the screen and buttons), and a flap that ‘opens’ your Pokédex.
The internals feature a XIAO ESP32S3 Sense microcontroller that has its own integrated camera, connected to a black and white OLED screen (based on the Pokédex toy from the 90s), an amplifier that hooks to a speaker, a set of breaker buttons, a battery, and a USB-C port for loading all the information to run the mini-computer, as well as to charge the battery.
The way the Pokédex works is rather clever – it uses GPT4 along with the PokéAPI, relying on the latter’s massive information database. The GPT4 gives the device its AI chops, and an AI voice generator (PlayHT) helps create the signature vocal effect of the Pokédex. Together, they work in tandem to first, identify the Pokémon, second, reference the information in the database, third, display the Pokémon on the screen, and finally, play relevant audio about the Pokémon’s name, type, background, and performance. This does, however, mean that the Pokédex needs to stay connected to WiFi at all times to constantly tap into GPT4 and the PokéAPI (since nothing happens locally on-device).
The entire process wasn’t without its fair share of problems, however. The problems started with the software itself, which hung, crashed, and sometimes got overburdened with just the amount of heavy lifting it had to do. Meanwhile, the PlayHT audio generator posed its own share of issues, like an annoying ticking noise that played in the background as the AI spoke. Abe mentions all the problems he had in a dedicated section of the video, also outlining how he fixed them (hint: a lot of coding).
Once all the bugs were fixed, Abe took his Pokédex out for a spin. In all fairness, it did a pretty good job of identifying Pokémon strictly by analyzing their shape. This meant the Pokédex worked absolutely flawlessly when pointed at images, or an accurate 3D figurine or toy. It didn’t however, fare too well with plushes, which can sometimes have exaggerated proportions. That being said, it’s still impressive that the Pokédex works ‘as advertised’.
Building your own isn’t simple, Abe mentions… although he does have a paywall on his YouTube page where paid members can get access to behind-the-scenes content where Abe talks more extensively about his entire process. If you’re a coding and engineering whiz (with a penchant for Pokémon and 3D printing), hop on over to the Abe’s Projects YouTube page and maybe you’ll figure out how to build your own Pokédex too! Maybe you’ll simplify the process so simpletons like us can build them as well…
The post Actual working Pokédex uses ChatGPT to identify Pokémon… and you can build one too first appeared on Yanko Design.
from Yanko Design
0 Comments