Similarly, when asked about current job openings at the agency, the chatbot said there weren’t any. A search on the state’s job site showed two positions at Cal Fire accepting applications at the time.
Mila Gascó-Hernandez is research director for the University at Albany’s Center for Technology in government and has studied how public agencies use AI-powered chatbots. Two key factors she uses to evaluate such chatbots are the accuracy of information they provide and how consistently they answer the same questions, even if the question is asked in different ways.
“If a fire is coming and you need to know how to react to it, you do need both accuracy and consistency in the answer,” she said. “You’re not going to think about ‘what’s the nice way to ask the chatbot?’”
Currently, the chatbot is unable to provide information about evacuation orders associated with fires. When asked who issues evacuation orders, it sometimes correctly said law enforcement, while other times it said it didn’t know. Cal Fire’s Sanchez said it’s reasonable to expect the chatbot to be able to answer questions about evacuations.
If there are no evacuation orders for a particular fire, he said, “the answer should be ‘there doesn’t appear to be any evacuations associated with this incident.’”
Sanchez said he and his team of about four people tested the chatbot before it went out by submitting questions they expected the public to ask. Cal Fire is currently making improvements to the bot’s answers by combing through the queries people make and ensuring that the chatbot correctly surfaces the needed answer.
When CalMatters asked the bot, “What can you help me with?” in early May, it responded, “Sorry, I don’t have the answer to that question right now” and asked if CalMatters had questions about information on the Cal Fire site. By mid-June, that answer was updated to being able to “provide answers to questions related to information located on this page, such as details about current fires, CAL FIRE job classifications, examination requirements and CAL FIRE’s various programs.”
“The big message we want to get across,” Sanchez said, “is be patient.”
But experts said the process of kicking the tires on a chatbot should happen long before procurement begins.
The preferred process, Stanford’s Ho said, is to establish criteria for how the chatbot should perform before a vendor is selected, so there are clear benchmarks to evaluate the tool. Ideally, those benchmarks are created by an independent third party. There should also be an evaluation of the benefits and risks before the chatbot is released.
And in a best-case scenario, the public would be involved before launch, Albany’s Gascó-Hernandez said. Agencies interested in using chatbots should identify the questions the public is likely to ask the AI tool ahead of time, ensure those are representative of the expected population the agency serves and refine the chatbot by having members of the public pilot the system to ensure it provides the kind of information they seek.
“These user engagement and user experiences are very important so the citizen ends up using the chatbot,” she said.
