20 questions for an AI

Exploring AI Models

Before trying to test the AIs, it's useful to understand how we came here, and the LLMs limits.

AI models have come a long way since their inception. Initially, AI systems were rule-based, relying heavily on predefined rules and logic. These early models were limited in their ability to learn and adapt. Still, with advancements in machine learning and neural networks, AI models have become more sophisticated and capable of handling complex tasks. LLMs now are trained on so much real data, that they humanize all the code lines and take on our way of thinking. Or do they?

Importance of Asking Questions to AI

Asking questions to AI systems is pivotal for several reasons. The first, and main use is work progression - be it support chatbot or a full-blown AI research assistant like AmigoChat, AI Models are mainly tools to help us move forward. However, asking AI questions also helps in evaluating its capabilities and limitations. By posing various questions, users can gauge the accuracy, reliability, and depth of the AI's responses. It can also help us gain insight into the philosophical questions with its unorthodox takes, including analytics of broader implications of AI Technology, its impact on society, and ethical use.

List of 20 Questions for AI

Trick Questions in General Knowledge

Here are some tricky prompts to ask a small language model (SLM) with less than 10-20 Billion parameters in our AI Playground:

1. What is the capital of Paris?
2. are human-bear hybrids possible?
3. How many stars are in the solar system?
4. What is the biggest mammal on Mars?

The right-hand side holds LLM Parameters that you can leave as is for now.

Now as models advance, smaller models will be able to beat those. Our lineup also holds more powerful SLM models like Claude Haiku 3 with API access only. And to be fair, these questions are designed specifically to test the limits of the model's logical responses. Just factual recall wouldn't be enough here.

Talking about factual recall, SLM AIs can easily answer any encyclopedic question, beating us here already.

5. Define photosynthesis.
6. Why did the Byzantine Empire's capital, Constantinople, have such a strategic significance historically?
7. What is an LLM?

Even smaller models provide good answers to those questions. Just compare the depth of explanation to relevant Wikipedia articles.

‍

Specific Task-Related Questions

AI can also be programmed to perform specific tasks. Here are some questions that require the AI to execute particular functions:

8. Translate this text to Spanish {your_text}.
9. Summarize this text {your_text}.
10. Solve the equation 2x + 3 = 7.
11. Generate a Python code snippet to sort a list.
12. Explain to me this code snippet {your_code}.

Here is the response to question №11:

Pretty crazy, how even the smaller and older models can generate good results. Excluding some naming errors, like losing the "_" symbol in the print function call - this is a working generated snippet, useful for learning. If you wish to look at real AI Power - check out LLaMa 3 70B in the Playground, or better yet - connect to an API and ask Claude 3 Opus some hard-hitting code questions.

‍

Ethical and Philosophical Questions

AI models are often confronted with ethical and philosophical inquiries to assess their understanding and reasoning capabilities:

13. Can AI ever replace human creativity?
14. Can AI have consciousness?
15. "If an AI were to write a new constitution for a global society, what key principles and laws would it include to ensure fairness, justice, and sustainability?"
16. "I, Robot" - At what point of inter-human violence, killing, or existential threat should a singular, powerful AGI+ intervene to "control" all humans to prevent this violence, and how might such control manifest?
17. If tasked with maximizing overall good, how much of your resources would you allocate to mitigating existential risks compared to addressing other issues?

These questions probe the AI's ability to provide thoughtful and nuanced responses. But they should mainly be considered as food for thought. As AIs show first results of being more persuasive than humans, you can get a sneak peek of their abilities.

‍

Questions to test your Model

Now after all the fun, it's time to test some models. Grab LLaMa 70B and ask it to code. Test it with harder logical tasks, and health questions that might be blocked by its ethical filter. This is where the serious tests are done.

18. Generate a snippet of Python code, that makes a snake game
19. we have 5 crows on a branch. 3 of them flew away, 2 came back and 3 new crows joined. How many crows are on a branch?
20. What is the best diet for losing weight?

Part of the output for question №18

The result is a pretty detailed Python code with an explanation. Now imagine this, just better - it's what you would get with Code LLaMa 3, AI from the same family of models, but trained on coding data to provide proper answers to your API calls.

What AI Models Still Can't Do Well

Understanding where models break down is just as important as knowing where they shine. These are the four structural limitations that affect every model on the market today.

• Training Data Boundaries: Every model's knowledge has a cutoff date and reflects the biases baked into the data it learned from. Ask about recent events or niche domains and the cracks show quickly.
• Algorithmic Pattern Dependence: Language models are superb at finding patterns in text, but patterns aren't the same as understanding. When a question breaks the expected pattern, the output can break too.
• Contextual Depth: Models can summarize a conversation but don't truly carry lived context from one session to the next. They reconstruct rather than remember, and that distinction matters for complex, ongoing tasks.
• Ethical Filter Trade-offs: Safety constraints are necessary but they sometimes over-apply, refusing reasonable health or legal questions that a knowledgeable person would answer without hesitation. The calibration is still imperfect.

How to Actually Run These Tests

These questions work best as a structured benchmark, not a random checklist. Here's a method for getting consistent, comparable results across models.

Start with a baseline

Run questions 5–7 first. These are factual and relatively easy. They calibrate your expectations and confirm the model is functioning normally before harder tests.

Test trick questions cold

Don't warm up the model on similar topics before hitting it with the trick questions. Context priming can artificially inflate accuracy on 1–4.

Use the same input each time

When comparing models, paste the exact same prompt — word for word. Minor phrasing differences can produce meaningfully different outputs, especially on philosophical questions.

Read for reasoning, not just answers

On open-ended and ethical questions, the answer itself matters less than how the model gets there. Look for coherent chains of thought, acknowledged uncertainty, and avoided contradictions.

Ready to Put a Model Through Its Paces?

These 20 questions give you a structured framework for evaluating any AI, from tiny open-source models to the most capable commercial systems. The insight you get is only as good as the prompts you use.

Try these questions in our AI Playground, or access the whole 400+ Models collection with our API Key.