Tuesday, May 21, 2024

Why does this happen in CrewAI only on local LLMs?

I am running the example crew that is created with the command crewai create newscrew, so, nothing too creative or experimental. When I run the query that comes installed in teh example, AI LLMs, It's work fine on OpenAI and on Ollama (both phi3 and llama3), but if I change the query to Bitcoin news, it still runs fine on OpenAI, but locally, I get a bunch of

Action: Research recent Bitcoin news post May 21st, 2023

Action Input: {
    "task": "Research",
    "context": "Gather information about significant developments in the Bitcoin industry from May 21st, 2023 to May 21st, 2024. This includes news articles, reports, and credible sources that provide insights into key events and their impacts on market dynamics, investor sentiment, and regulatory environment.",
    "co-worker": "Bitcoin News Senior Data Researcher"
} 

Action 'Research recent Bitcoin news post May 21st, 2023' don't exist, these are the only available Actions:
 Delegate work to co-worker: Delegate work to co-worker(task: str, context: str, coworker: Optional[str] = None, **kwargs) - Delegate a specific task to one of the following co-workers: [Bitcoin News Senior Data Researcher
]
The input to this tool should be the co-worker, the task you want them to do, and ALL necessary context to execute the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(question: str, context: str, coworker: Optional[str] = None, **kwargs) - Ask a specific question to one of the following co-workers: [Bitcoin News Senior Data Researcher
]
The input to this tool should be the co-worker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.

The problems seem to be related to the answers CrewAi is getting back from the requests. For example...

Thought: To provide an in-depth analysis of each event mentioned in the news headlines, I will gather information from various sources such as financial reports, expert opinions, and regulatory documents to ensure a comprehensive understanding of their implications on cryptocurrency regulations and global finance.

Action: Search
Action Input: "IMF report on Bitcoin impact" 

Action 'Search' don't exist, these are the only available Actions:

Some 'thoughts' in particulr trigger a Action to call a non-existane co-worker that not defined anywhere (and allow_delegation=False).

Is there any way to stop this? Like, can it just look for co-workers that are defined as existing? Or am I not understanding the issue?

BTW, these errors do not occur when I switch to OpenAI (gpt4)... so, does this suggest the problem is in LLMs?


No comments:

Post a Comment