{"id":2105,"date":"2024-05-29T17:44:14","date_gmt":"2024-05-30T00:44:14","guid":{"rendered":"https:\/\/www.digivideoplus.com\/dv_blog\/?p=2105"},"modified":"2024-06-27T16:55:46","modified_gmt":"2024-06-27T23:55:46","slug":"generative-ai-sans-horse-sense-unreliable-narrators-by-design","status":"publish","type":"post","link":"https:\/\/www.digivideoplus.com\/dv_blog\/2024\/05\/29\/generative-ai-sans-horse-sense-unreliable-narrators-by-design\/","title":{"rendered":"Generative AI sans horse sense \u2013 unreliable narrators by design?"},"content":{"rendered":"\n
\"\"<\/a><\/figure>\n\n\n\n

I like the way that I can do a Google search using a photo \u2013 drag & drop it into a search box \u2013 to identify its content. I like the way I can extract text from a photo (providing the photo has sufficient resolution on that content). And Amazon uses AI to summarize the gist of product reviews (to some degree).<\/p>\n\n\n\n

But in the rush to embed AI for directly answering user questions \u2013 in a manner like “Ask Mr. Wizard<\/a>,” this article notes that something is amiss. With large language models. Perhaps with no way out.<\/p>\n\n\n\n

Not all movies can be saved in post. Not all software can be saved by updates. GIGO<\/a>.<\/p>\n\n\n\n

Rolling the dice is not a good basis for trustworthiness. “Mostly correct” incurs hapless harms. A form of quantum uncertainty: horse sense or horse pucky, eh.<\/p>\n\n\n\n

Who vets fact-checking mechanisms? And if it comes to using low-wage human labor to fact-check, \u2026<\/p>\n\n\n\n

\u2022 Washington Post > Tech Brief > email news > “Google’s AI search problem may never be fully solved” by Will Oremus (May 29, 2024) \u2013 Last week, Google’s new “AI Overviews” stretched factuality.<\/p>\n\n\n\n

\n

“All large language models, by the very nature of their architecture, are inherently and irredeemably unreliable narrators,” said Grady Booch, a renowned computer scientist. At a basic level, they’re designed to generate answers that sound coherent \u2014 not answers that are true<\/strong>. “As such, they simply cannot be ‘fixed,'” he said, because making things up is “an inescapable property of how they work.”<\/p>\n\n\n\n

But that [citing and summarizing specific sources] can still go wrong in multiple ways, said Melanie Mitchell, a professor at the Santa Fe Institute who researches complex systems. One is that the system can’t always tell whether a given source provides a reliable answer to the question, perhaps because it fails to understand the context. Another is that even when it finds a good source, it may misinterpret what that source is saying.<\/p>\n\n\n\n

Other AI tools \u2026 may not get the same answers wrong that Google does. But they will get others wrong that Google gets right. “The AI to do this in a much more trustworthy way just doesn’t exist yet,” Mitchell said.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"

I like the way that I can do a Google search using a photo \u2013 drag & drop it into a search box \u2013 to identify its content. I like the way I can extract text from a photo (providing the photo has sufficient resolution on that content). And Amazon uses AI to summarize the gist […]<\/p>\n","protected":false},"author":2,"featured_media":1900,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,6],"tags":[192,242,241,243,244],"class_list":["post-2105","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-computer","category-research","tag-chatbot","tag-facts","tag-generative-ai","tag-search","tag-trust"],"_links":{"self":[{"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/posts\/2105","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/comments?post=2105"}],"version-history":[{"count":3,"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/posts\/2105\/revisions"}],"predecessor-version":[{"id":2108,"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/posts\/2105\/revisions\/2108"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/media\/1900"}],"wp:attachment":[{"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/media?parent=2105"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/categories?post=2105"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.digivideoplus.com\/dv_blog\/wp-json\/wp\/v2\/tags?post=2105"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}