When Not to Use Gen AI

👉 Surfing around the hype

👉 LLMs and what they are not for

👉 Gen AI solves my problem, can I use it?

👉 Some good use cases for Gen AI

👉 Wrapping up

Surfing around the hype

A bit of foreshadowing

Recent progress in AI has brought an almost mystical view to the general public about what is possible with current technology. Coders and other technical people are no exception to this phenomenon. This is not a bad thing; I’d even argue that it's a good thing.

On the industry side of things, the situation is quite similar. Everyone wants to jump into the Gen AI bandwagon, even when other standard or simpler solutions are available. This may happen for a variety of reasons: lack of in-depth knowledge, lack of well-studied problems, or plain old hype. This mentality may hinder business in more ways than you realize, and you may miss out on the actual benefits of this new tech. So, in this article, we present: When (not) to use Gen AI? The dos and don’ts of using generative AI where it truly adds value.

LLMs and what they are not for

LLMs (Large Language Models) can be used in a variety of ways for multiple applications. They can code, write ads, devise action plans for complex sequences of operations, and even answer your phone call. However, they should not be used as know-it-all oracle machines, especially for simpler problems. Here are some examples:

• Don’t use AI language models like ChatGPT to categorize customers. Established data-driven methods, based on your data, are more reliable and better suited to capture specific customer needs and local nuances. ChatGPT relies on general trends, which can lead to misguided decisions without tailored tuning.

• Avoid using generative AI tools to make predictions, like setting an ideal price range for products. These tools aren’t designed for that. Traditional data analysis methods, such as established statistical models like Random Forest or optimization methods, are still reliable and might be a better fit for your needs.

LLMs are also really expensive as they require quite a lot of compute compared to more traditional methods. Does it make sense from a business standpoint to use this technology?

Gen AI solves my problem, can I use it?

Yes! But is it the best solution? Is it the cheapest? You may realize that your problem can be solved in a better way by more traditional methods. Here are some real examples:

Using Gen AI to detect objects

Take, for instance, object recognition in a video. New fashionable approaches involve using multimodal LLMs to detect things in videos. These types of models are incredibly flexible; they can be used for complex scene understanding. However, if you are just concerned about, let’s say, detecting people in a video, this is not ideal. The model will sort of do it, but it will be slower, less predictable, and more expensive.

Let’s say, for example, you use GPT-4 API calls for a video at 512x512 resolution and 30 fps. Processing every frame would cost around $2,20 per minute. On the other hand, if we use more traditional approaches for object detection, we get a cost (using a rough estimate) of around $0.0007 per minute (3,000 times cheaper per minute—five orders of magnitude). We could use some tricks to reduce costs, such as processing every nth frame, but that is not the issue. **We must use the right tool for the problem.**

Using LLMs as straightforward "if A, then B" decision bots

A very natural use case of LLMs is as bots. This is a great use case; however, LLMs have their own set of problems that you may need to take into account. They are unreliable, need guardrails, have high latency, and are expensive to run on a massive scale. Their greatest strength can also be their weakness: their flexibility. Traditional bots of the sort "if A, do B" might be exactly what you need! In many uses, you don’t actually need or want your bot to handle every possible user request. Things such as hotel booking are still plain and simple old-school bots. No need to reinvent the wheel. These are fast, easy to implement, and you have guaranteed behaviors. Note that it’s possible to mix traditional and LLM-based bots to get the best of both worlds.

Some good use cases for Gen AI

So far, we've reviewed some not-so-ideal cases for Gen AI; however, there are some exciting possibilities that open up because of these tools. Here are some actually reasonable use cases for Gen AI. Of course, these are just some business cases we’ve seen. This should not limit you from trying out new use cases. We’ve already talked about the future in a recent post. Having said that, here are a couple of examples just to convey what I'm talking about:

Querying datasets and large knowledge bases using natural language

LLMs are really good at processing and structuring large amounts of text. For instance, you can combine traditional database search methods with LLMs, allowing customers to interact with their data using natural language without additional training. A good example of this is Amazon's Rufus, a bot that helps customers learn more about their products and do better searches. Older methods cannot do this, or at least not at this quality and flexibility.

Automatic content generation

Ads—both text- and image-based—are clear candidates for generative AI, enabling capabilities that older methods couldn’t achieve. This includes creating ad variants with specific messaging or localizing content for different regions. Image editing also benefits significantly from generative AI, allowing for brand-aligned advertisements to be created automatically for your products.

Complex situational understanding

Older methods could roughly get the gist of some media. Sentiment analysis, for instance, allows you to classify the direction a piece of text leans concerning a topic such as a political agenda.

There are a lot of simple and complicated algorithms for doing this. However, when it comes to subtlety, modern Gen AI techniques are king. If your use case requires understanding small things which involve some kind of world understanding, then LLMs and the like might be for you.

Gen AI might help when you don’t have enough data

Here’s a particularly fancy use of Gen AI (particularly LLMs). Suppose you are in one of those cases where a traditional approach would work. However, what if you do not have the required data for it to work well or at all? Gen AI models leverage a huge amount of general knowledge, and while not specific to your application, it might have connections to it. Such tools can be used to generate synthetic data for other models. We are only using an LLM as a tool for training, and we would still use our cheap and fast models for inference. We can also bite the bullet and directly use a pre-trained LLM as the solution for our task, at least until we get more data.

Wrapping up

Modern methods promise good results with little or no training data at all, but they are not (yet) a superior solution to standard methods for all usual problems. They are not yet the state-of-the-art replacement for our bag of tricks but more of a new tool to incorporate.

Hopefully, this article will help you browse beneath the hype and see the breakthroughs. We should use Gen AI when it's better than previous techniques or for a new case that is impossible without it. After all, new tools should always stretch what we can do, not shrink it.