Safeguarding your customer’s data: Exploring ethical AI in data privacy

Oct 13th 2023ai

The end of 2022 marked a major milestone for the AI world: the release of OpenAI’s ChatGPT. Worldwide attention was drawn to this new generation of models as everyone could test the chatbot’s impressive capabilities. With the advent of companies like HuggingFace that allow anyone to leverage pre-trained models, building with AI is now easier than ever. As a result, AI is clearly gaining a lot of traction. However, if there is one area that doesn’t get as much attention, it’s ethics, and particularly ethical AI. 

This lack of focus around ethical AI can have far reaching implications for how the industry will develop. The reason for this is that the next wave of AI technological development will leverage streams of personal data – from social media to devices – at an unprecedented scale. As machine learning becomes more prevalent from analyzing and predicting to informing and decision making in almost every facet of our daily lives – unexpected privacy and policy issues will certainly arise. To safeguard the interests of companies, researchers, and most importantly, consumers, AI needs to be greater than just data and algorithms. On the path to product, it is imperative that companies choose the ethical route.

Data Privacy

In 2006, long before whisperings of AI, AOL intentionally released the private search history for 20 million queries for 650K users for research purposes. Though they replaced the usernames with random numbers, no additional effort was made to filter the search queries of private or sensitive data.

Though AOL quickly took down the site, the damage was already done. The file containing their customers’ privacy-sensitive data was already in the wild and being shared. It took only three days for The New York Times to identify users through their unfiltered search queries and release a scathing article. Consequently, the fiasco cost the jobs of three AOL employees including the CTO as well as a catastrophic reputation hit to an already struggling company.

The debate over digital data privacy has been ongoing for decades with proponents arguing for greater protections such as the right of removal of personal information from the internet. Though notice-and-choice has been the historical model for privacy regulations, the obligation is shifting towards the companies that collect data. Current trends lean towards more actively regulated data privacy with the passage of legislation like the General Data Protection Regulation (GDPR) in Europe. Though regulations like the GDPR, the California Consumer Privacy Act (CCPA), and Health Insurance Portability and Accountability Act (HIPPA) may impose penalties and seek remediation from companies that violate consumer privacy, regulatory agencies can’t mitigate the damage once private data has been leaked. The internet never forgets, and once the data is made available there is no way to put these “devices” back into the proverbial Pandora’s box.

There is also the potential of incorrect or misleading information. For example, a language model learns from user queries and someone types “cancer cure vinegar.” The next user types “cancer cure” and the language model may update autocomplete the query with “vinegar.” 

These issues were particularly apparent when on November 15, 2022 Meta released its flawed Galactica LLM trained on 48 million examples of scientific articles, websites, textbooks, lecture notes, and encyclopedias. Originally developed to assist researchers and students, the AI generated wrong or biased results, including a wiki article about the “history of bears in space.” After three days of criticism, Meta shut down its public demo.

Any company using user data to train their AI is exposed to privacy-sensitive data risks. If unfiltered ML training data is used to construct an AI’s large-language mode (LLM), the AI could generate public results containing personally identifiable information (PII). 

PII Filter © Algolia

It is imperative that AI companies protect their private consumer data by following good governance, best practices, and verified protocols that are scrutinized to ensure PII is handled correctly. 

To ensure data privacy, ethical AI emphasizes the following principles:

• Transparency: customers should have notice of their data rights, access to their information, and control over how their data is shared. 
• Responsibility: companies should implement governance and tools that evaluate data privacy risks.
• Accountability: companies should consistently audit and monitor the impact of their AI systems throughout the lifecycle. 

Further, data collection should not be continuous, only permissible under certain conditions, and retention should be limited. When deploying an AI system, companies should use the minimum data set required to reach their goals and not go beyond what is absolutely necessary. 

For example, if you’re collecting customer geographic data how will it be used? If the data trends are used solely for the company’s sake without any benefit to the customer (e.g. to analyze sales trends), it could be considered borderline exploitative. Ideally, you want the collected data to actually help the customer such as create a better personalized experience (e.g. suggest popular movies in their country). Collected data should bring value to the customer, not just the company.

In the race to collect more data and scale their AI models, companies run the risk neglecting data privacy. Having a data and AI ethics strategy reduces the chance of a major data incident that could expose a company to reputational damage, financial consequences, and legal implications.

Trust is the foundation of your customer relationship

Your customer’s trust is hard to build, easy to destroy, and even tougher to repair. To maintain trust between the AI and the customer, clearly communicate your data usage policies and employ “privacy first” practices when designing, developing, and deploying AI models. Likewise, an organization’s staff, internal policies, and processes should provide data rules of engagement and ample oversight to ensure AI systems use data ethically and responsibly.

At Algolia, we put securing customer privacy and data at the heart of all of our solutions. To learn more contact one of our search experts today or get a customized demo.