Maurice Heriveaux, MBA

Let me tell you a short story about AI in HR Tech - it's about the power of frontier LLM's and how access to a proprietary or specialized data set may not actually be as beneficial as you'd assume. Once, not too long ago, I gained access to an AI solution that was being sold by a company that had years of specialized and proprietary data. Let's call it Solution A. Solution A, which appeared to be powered by ChatGPT, presumably would benefit from having access to the company's proprietary data/information - this is why I was specifically interested. During some testing, I asked Solution A and the public version of ChatGPT the same questions that were specifically in the domain of the proprietary data/information of the company behind Solution A. Then I asked Solution A which answers were better. Solution A answered that ChatGPT's answers were better, and explained why. Of course, I didn't just take Solution A's response at face value - I used my critical thinking to evaluate the responses and I happened to agree. I found this funny at first, and a bit ironic. However, as I thought about how this could be - the answer is actually quite obvious. While there is certainly value in curated proprietary data sets, it is not safe to assume that the value of those data sets, and the insights that can be gained from them, is more valuable or insightful than the collective value/insights that a frontier model like ChatGPT has as a result of the information it was exposed to/trained on. The frontier models have essentially "read the entire internet" - billions of documents, research/white papers, forum discussions, case studies, and expert analyses across every domain imaginable. That's a lot of collective human knowledge and wisdom baked in. What does this mean? When evaluating AI solutions that use proprietary data access as their key differentiator, test them against the public frontier models. Ask the same questions. Compare the outputs. Ask the proprietary solution itself which answer is better - you might be surprised by the response - and then judge for yourself. This doesn't mean specialized solutions lack value - especially if you are specifically interested in the data set they offer. But don't assume that proprietary data automatically equals superior insights. Why ask only one company for their insights when you can ask many? The frontier models might already know more about your specialized domain than you'd expect - because somewhere in their training data, experts may have already discussed, debated, learned from, documented, and shared the very insights you're seeking. Of course, this isn't an either/or scenario - you can purchase access to a proprietary solution AND use other free or paid AI...more models are better than one. Why not ask a mixture of experts? The moral of the story? Assume less. Question and test more. Use your critical thinking. 🧠 #AI #HRTech

43

9 Comments