When we first built GPT-3 into Cheat Layer, the idea was we could keep adding training data based on what users want. Eventually, that process would build a model which was functionally a software engineer who specialized in automations and integrations. While the model didn’t translate all the jargon yet, we deployed two colliding solutions to that problem:
1) Teach the model what users are asking and pair it to complete solutions, so over time the model translates all the jargon.
2) Teach users what the model expects in language patterns through onboardings, bootcamps, and guides.
Through our bootcamp, we noticed two practical road blocks to this:
1) Training beyond the prompt with GPT-3/Codex requires hundreds of examples for each use-case, and this takes an immense amount of time to gather. We’re lucky to gather 1 example for the best use-cases.
2) Making people ‘remember’ what to say doesn’t actually solve the original problem, which is to abstract away all the UI complexities of services so users can just say “Do this” and it magically works. That’s the holy grail.
We often got support requests of complex phrases that didn’t work, and we explained the model isn’t perfect yet, but in our vision those should all absolutely work.
So we thought about how to solve the problem completely faster using Codex and everything we knew.. The solutions were: general AI, an army of coders, or leveraging Codex/language models plus machine learning to abstract the details on both ends and meet all users at their level of understanding.
In this next major update of Cheat Layer Desktop, we’re automating the detection of UI components using machine learning, then abstracting that in the Codex language layer to formalize automating the UI for all the top websites and programs.
For example, we can say
“Go to Amazon, find this product, and click the button with Buy Now in it”
with AI object detection, and then take that one step further with Codex to abstract away jargon and meet all users at their level of understanding:
“Find this product on Amazon and buy it”.
