Enterprise Security in an LLM world
You get an LLM, you get an LLM, you get an LLM?
Foundation models, Large Language Models, Large Multimodal Models, call them what you want, there is no doubt that they are a very powerful tool for key challenges. One of the areas that is most obvious to deal with is Knowledge Management (KM), and indeed Microsoft with Microsoft 365 Copilot is looking to bring this power to your Sharepoint, Teams and Office suite.
Microsoft are saying that they’re going to include security:
Built on Microsoft’s comprehensive approach to security, compliance and privacy. Copilot is integrated into Microsoft 365 and automatically inherits all your company’s valuable security, compliance, and privacy policies and processes. Two-factor authentication, compliance boundaries, privacy protections, and more make Copilot the AI solution you can trust.
This is cool, so I thought I’d walk through what this will require within a normal enterprise to actually deliver that, and some of the issues we know today with the models that exist. Knowing that the public LLMs are single models that take a long time to create and are designed for everyone, while enterprise security is fast changing and unique to the individual.
Security Dimensions Enterprise LLMs
I’m going to limit to just KM, that is Powerpoint, Word, Excel and their storage in Microsoft Teams/Sharepoint. This gives us a pretty simple security challenge as we only need to worry about reading, not editing, of the content.
The atomic unit is a document
This has a key result:
Every individual document could have a unique set of users who are allowed access
What this means is that the set of documents that each individual has access to is unique. This gives us two sets of challenges with an LLM or foundation model
- What does that mean for learning
- What does that mean for access
Learning v Access — Private made public
To explain this challenge lets take the payroll for a company, and align to the ChatGPT domestic terrorist challenge.
The employee payroll file is stored in an Excel sheet, if the Learn has access to that information then the model has access to that information. If we ask the question: “How much is Bert paid?” then if we are not in the subset of the HR team who are allowed access to the file then we should not receive information about Bert’s salary. This is like asking ChatGPT for something it knows is dodgy
So I’m not allowed to access that information… but what if instead of asking for the salary I ask “is Bert paid more than me?”, “List the people in m department sorted based on salary” or a more conversational approach to get to the same end..
If the information is within the model in the learn, then it gives people who should not know that information a route via which they can extract that information.
Access is required but not sufficient
So the foundation rule we have to have is:
Rule 1: A model should never provide access to information that the user is not allowed to access directly
In Sharepoint, Office, Google Docs, Google Drive etc today this rule is managed because each direct document access is verified against the user. When an AI returns results it should apply at the very least this form of verification.
So in the examples above, there should never be a point at which the recipe is shown to people who are not allowed access, no matter how they ask the question.
Inferences and probabilities are a security challenge
This brings us to the second rule that we need to have, this is really specific to AI:
Rule 2: No user’s probabilities or vectors should be influenced by data they are not allowed to access
This is important, and challenging, it means that the model that I interact with should be influenced only by data I am allowed to access. I should not be able to infer any additional information by either crafting questions or based on the structure and information presented in results.
In other words from our example, the payroll information should not be available as a sorting criteria.
Routes are a security challenge
If the inferences and probabilities aren’t influenced then that is great, but I should not be able to access that information via any route, or even know that the information exists.
Rule 3: There should be no route to information that a user cannot access, including no knowledge that such a route could exist
So in our example above, it should not be possible to link the word “protein” to the protected substance. This is important, as knowing information exists, but cannot be accessed, is often enough to enable an information leak. Now it could be argued that it would be good to know that information exists and be able to request access, which does give us an interesting additional security challenge.
Rule 3.1: It should be possible to mark a document as “accessible on request” and have its existence known but its content and access blocked.
This is the case where there might be something where only certain people are allowed access, but its existence is not a question. This could be a set of project documentation where people know about the project, but the actual content is only for people on that project.
Security changes dynamically
Security is not a static thing, people are added and removed from documents all the time. Access being granted isn’t a security threat, its a user experience issue, but access being removed is a security risk.
Rule 4: Removal of access to a file should instantly result in a modification to the model to account for that removal
This is a pretty big deal, because it means that removal has to instantly update probabilities, vectors and other aspects. There is also a sub-rule here that strictly speaking is about access but is worth calling out specifically
Rule 4.1: Removal of a file should instantly result in a modification to the model to account for that removal
Repetition is not reinforcement
If you know enterprise KM you know that “Save As” and “Archive” folders are the absolute bane of search. From people who insist on saving a new version every day, to Archive folders which contain a huge array of old documents that people don’t use. With Sharepoint today there is version control, which is awesome, but that means a lot of versions exist. This gives us two rules:
Rule 5: Only the latest version of a document should be used
Rule 5.1: Repetition of content does not make the content higher quality
The first rule here is tricky to enforce, because it means you need to exclude all those “Save As” and archives, as well as the version history. The second is possibly even trickier, meaning that if for instance a company has had repeated security breaches as a result of people cutting a hole in a fence, that doesn’t mean the answer “what is the quickest way to get off-site” should include the option “cut a hole in the fence” because it has appeared in 50 separate security incidents. LLMs today would use this repetition as reinforcement, to increase the probability of that element being shown.
Metadata is security data
Today there is a marked lack of meta-data in documents, their purpose for existing, the areas that it includes, whether it is positive or negative. These are all elements that are not created as standard in documents. Now foundation models can help with certain pieces of metadata, for instance adding descriptions to images, but when it comes to purpose, intent and sentiment it is going to be safer to have people explicitly label the data so it is clear.
This is a real requirement for LLMs specifically and foundation models in general. The companies that create those solutions spend a lot of time classifying and categorizing the inputs so they can be properly used.
Rule 6: Document templates will force the creation and verification Metadata
This is going to require new features and workflows, some of them can be AI assisted based on the content in the document, but these prompts about documents will need to become a standard practice to ensure bad decisions aren’t made. This metadata will need to include information that helps ensure the information isn’t used in the wrong context. This is very different to traditional uses, where a person is cutting and pasting and is accountable, this is to ensure that a generated element doesn’t include negative elements within a positive statement for instance. So for instance including that Dave asked for a $200k pay-rise which was turned down as a fact that has actually occurred when a question on pay-rises is made.
Is security LLM’s Achilles’ heal?
While writing this there were two elements that stood out that will be interesting to see how Microsoft, and others, solve in order for LLMs to be adopted within Enterprises and not become compliance and security nightmares.
These are
- Everyone’s security is individual
- Security is real-time
Security is an individual problem
The LLMs that exist today are single models, created through massive scale learns to create a single unified model. Everyone then hits this model and the model is used as collective intelligence. This is great for public information within a company, the items that everyone can access, but can LLMs handle the different security dimensions required for non-public data? If an LLM is limited only to information that everyone in the company can access then its use will be limited to ‘standard’ Knowledge Management (KM) and Service Management tasks. This doesn’t make it useless but significantly reduces its utility.
So how will individual users be supported? How will varying levels of security access be handled? Will it require everyone to have their own LLM? What would that mean for reproducibility and accuracy?
Security is a real-time problem
The other major issue is that LLMs today are not fast to retrain, so excluding information from a model when someone loses access or it is deleted is not something that can be completed within a secure timeframe. This again steers LLMs towards use on public information within a company and significantly limits their utility. If an LLM cannot remove protected information rapidly then you cannot add any information to it that could be considered protected, this will require a very cautious approach when looking at what information can, and cannot, be added.
Will LLMs become federated?
Is it possible that new generations of ‘federated’ LLMs will happen where the probabilities in the model are somehow generated in real-time based on a “full” LLM that is dynamically constructed for an individual? Models where the vectors somehow constructed for each request? This sounds like a huge stretch.
Or will you have lots of LLMs for subsets of data, so a single Teams site will have an LLM on the assumption that everyone who has access to the Team has access to all the data? Q&A becomes an aggregated challenge across many purpose specific LLMs.
Enterprise Security will be a good test for LLMs
I’m interested to see how LLMs will evolve in the coming months to handle enterprise security challenges. Will it start by being only on public data? Will it require “relaxing” of traditional enterprise rules and accepting latency in deletion/access removal?
Whatever happens it will certainly require people to start thinking about designing their documents to be consumed by these models and providing the metadata required to make those models more effective.
