The conversation

firefoxdHNSource author
Business & incomeExperience excerpt · English

I worked on an Ai for customer service

… I worked on an Ai for customer service. Our agent took the average response time of 24/48 hours to merely seconds. One of the messages that went to a customer was "Hello Bitch, your package will be picked up by USPS today, here is the tracking number..." The customer responded "thank you so much" and gave us a perfect score in CSAT rating. Speed trumps everything, even when you make such a horrible mistake.

Original discussionShow HN: Voice bots with 500ms response times99 source comments · at import
3

What was this responding to?

Parent context · original excerpt

KW
kwindlaHN

Last year when GPT-4 was released I started making lots of little voice + LLM experiments. Voice interfaces are fun; there are several interesting new problem spaces to explore. I'm convinced that voice is going to be a bigger and bigger part of how we all interact with generative AI. But one thing that's hard, today, is building voice bots that respond as quickly as humans do in conversation. A 500ms voice-to-voice response time is just barely possible with today's AI models. You can get down to 500ms if you: host transcription, LLM inference, and voice generation all together in one place; are careful about how you route and pipeline all the data; and the gods of both wifi and vram caching smile on you. Here's a demo of a 500ms-capable voice bot, plus a container you can deploy to run it yourself on an A10/A100/H100 if you want to: https://fastvoiceagent.cerebrium.ai/ We've been coll…

Full original

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 3

Direct replies available in this import. Original-language excerpts.

LU
lukanHN

"The customer responded "thank you so much" and gave us a perfect score in CSAT rating. Speed trumps everything, even when you make such a horrible mistake." I think not everyone would react the same way. For some calling each other bitch is normal talk (which is likely, why I it got into the training data in the first place). For others, not so much.

Full original
FI
firefoxdHN

Fun fact, we fixed this issue by adding a #profanity tag and dropping the message to the next human agent. Now our most prolific sales engineer could no longer run demos to potential clients. He had many embarrassing calls where the Ai would just not respond. His last name was Dick.

Full original
BE
bedel23HN

I wonder if the solution is to run the message through another LLM to make the message as polite as possible removing any profanities. Cost >2x as much to run though.

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.