Updated 8 October 2026: Microsoft is retiring Restricted SharePoint Search, so this post now points to Restricted Content Discovery and SharePoint Advanced Management instead.
A Microsoft 365 Copilot pilot program that is properly scoped and governed will tell you exactly what controls you need before deploying to your whole organisation — and will surface the data governance gaps that, if left unaddressed, create real privacy and security risk at scale.
Running a pilot without the right preparation is not a safe middle ground. Copilot queries every SharePoint site, OneDrive folder, and Teams conversation the signed-in user has read access to. In an ungoverned Microsoft 365 environment, that means the pilot itself becomes the incident — not the controlled learning exercise it should be.
This guide gives Australian IT practitioners a structured approach to planning, executing, and evaluating a Copilot pilot that is genuinely useful, defensible under the Privacy Act 1988, and aligned with the ACSC Essential Eight.
Step 1: Complete a Readiness Assessment Before You Touch Copilot
The biggest mistake Australian organisations make is treating a Copilot pilot as a product trial rather than an AI deployment project. Copilot is not a standalone product — it is an AI layer over your existing Microsoft 365 data. The quality and security of that data determines the quality and risk profile of every Copilot response.
Before assigning a single licence, complete a structured Copilot deployment readiness assessment that covers:
- Identity and access controls — Are all users covered by MFA? Is Conditional Access enforcing compliant device requirements?
- Data classification — What percentage of SharePoint content has sensitivity labels applied? Where is personal information stored without classification?
- Permission sprawl — How many SharePoint sites have "everyone except external users" sharing? How many guest accounts exist with no expiry?
- Secure Score baseline — What is your Microsoft Secure Score today? Which recommended actions are directly relevant to Copilot governance?
Document every finding. Your readiness score becomes the benchmark against which you measure pilot outcomes.
Step 2: Define Your Pilot Scope and Participant Criteria
A well-scoped pilot is not "whoever wants to try it." It is a deliberate cross-section of your organisation selected to generate meaningful signal about Copilot's value and risk profile.
Participant selection criteria
Select participants across at minimum three to four different business functions. Include:
- Power users — staff who write, summarise, or research heavily and will get measurable productivity gains
- Risk-sensitive roles — HR, finance, or legal staff who work with confidential information, giving you signal on whether governance controls are working
- Technical users — IT staff who can provide detailed feedback on integration behaviour and surfacing accuracy
- Sceptics — at least a few participants who are not enthusiastic about AI; their feedback often surfaces real usability and trust gaps
Exclude staff who handle information subject to legal professional privilege or statutory secrecy obligations (certain government roles) until legal advice confirms Copilot use is permissible for those data types.
Scope the data boundary
Define which SharePoint sites, Teams channels, and OneDrive content Copilot is permitted to query during the pilot. Enforce this technically with SharePoint Advanced Management (SAM) reports, site access reviews, Restricted Content Discovery and Restricted Access Control — do not rely on user discretion alone.
Step 3: Implement the Mandatory Security Controls
These controls are non-negotiable for any Australian organisation running a Copilot pilot. They reflect the minimum reasonable standard under the Australian Privacy Principles.
3.1 Review oversharing and protect sensitive sites
Run SAM data access governance reports for the sites in your pilot scope. The reports cover site permissions, sharing links and "Everyone except external users" access. Delegate site access reviews to the site owners and fix the findings before pilot participants get licences.
Where a sensitive site cannot be fixed before the pilot starts, apply Restricted Content Discovery to it. This keeps the site's content out of organisation-wide search and Copilot responses without changing permissions. Restricted Access Control limits a site to members of a named security group or Microsoft 365 group. Treat Restricted Content Discovery as temporary: excessive use reduces the completeness of Copilot answers, so plan to remove it once access is remediated. See Microsoft Learn — Restricted Content Discovery.
Restricted SharePoint Search, which limited Copilot to an allow list of sites, is retiring and new enablement has been blocked since 31 July 2026. If your tenant already uses it, plan to disable it after you have validated permissions.
3.2 Apply sensitivity labels to pilot scope content
Before the pilot starts, run a Content Explorer sweep of SharePoint sites in your pilot scope. Identify documents containing personal information, financial data, or commercially sensitive content that lack sensitivity labels. Apply Auto-labelling policies to catch unclassified content automatically.
3.3 Configure Data Loss Prevention policies for Copilot
In Microsoft Purview, create or extend DLP policies to:
- Block Copilot from summarising content labelled "Highly Confidential" outside an approved context
- Generate audit logs every time Copilot interacts with content labelled "Confidential" or above
- Alert your security team when a pilot user attempts to use Copilot to extract information matching personal data patterns (TFN, Medicare number, passport number)
3.4 Enable Microsoft Copilot usage reporting
In the Microsoft 365 Admin Centre, enable Copilot usage reports. You need this data to assess adoption patterns and to identify anomalous behaviour during the pilot — for example, a single user running an unusually high volume of Copilot queries against HR documentation.
Step 4: Define Success Metrics Before You Start
A pilot without pre-defined success metrics produces opinions, not evidence. Define your metrics at the start and collect baseline data so you can compare objectively at the end.
Recommended metrics for a 90-day Copilot pilot:
| Metric | How to Measure | Target |
|---|---|---|
| Active Copilot users per week | Microsoft 365 Usage Reports | Over 70 percent of pilot cohort by week 8 |
| Secure Score improvement | Microsoft Secure Score dashboard | Measurable increase from baseline |
| Security incidents related to data exposure | Security Operations ticketing system | Zero incidents attributable to Copilot |
| IT support tickets related to Copilot | Helpdesk system | Declining trend after week 4 |
| User satisfaction score | Fortnightly pulse survey (5-point scale) | Average 3.5 or above by end of pilot |
| Sensitivity labelling coverage | Microsoft Purview Content Explorer | Improvement versus pre-pilot baseline |
Document these baselines in writing before the pilot begins. You will need them for the business case that supports or declines full deployment.
Step 5: Build a Structured Feedback and Governance Loop
Copilot pilots that do not include a structured feedback mechanism produce anecdote rather than evidence. Build the following cadence into your pilot plan from the start:
Week 1-2: Onboarding and baseline capture
Run a one-hour onboarding session for all pilot participants. Cover what Copilot can and cannot access, the data classification controls you have implemented, and how to report a concern if Copilot surfaces something unexpected. Capture your Secure Score baseline on day one.
Week 4: First security review
Pull Copilot audit logs and DLP alerts. Check whether any alerts fired during weeks one through three. Review the SAM data access governance reports and your Restricted Content Discovery settings — are there any sites participants are asking Copilot about that should go through a governance review first? Adjust controls as needed.
Week 8: Mid-pilot governance checkpoint
Run a structured review with your security team, a business representative from the pilot cohort, and your Copilot licence owner. Assess: Is the pilot generating the expected productivity signal? Are any governance controls creating friction that is reducing adoption? Are there data hygiene issues that were surfaced that need remediation before full rollout?
Week 12: Pilot completion assessment
Complete a formal assessment against your pre-defined success metrics. Produce a written recommendation — deploy, extend the pilot, or halt — with evidence. This document becomes the basis for your board or executive briefing on Copilot deployment.
If you are preparing for full deployment after a successful pilot, the Microsoft 365 Copilot Wave 3 Deployment Playbook for Australia covers the full enterprise deployment pattern including tenant configuration, change management, and ongoing governance cadence.
Step 6: Manage Change Alongside the Technical Deployment
The most common reason Copilot pilots fail to convert to full deployment is not a technical problem — it is a change management failure. Users who do not understand what Copilot is doing, or who do not trust it, will not use it. And a tool with low adoption does not generate the business case for enterprise rollout.
Practical change management steps for Australian organisations:
- Communicate the "why" in terms users care about — explain how Copilot will reduce their administrative burden, not just the technical architecture
- Acknowledge the data concerns directly — Australian employees are increasingly aware of AI privacy implications; address what Copilot can and cannot see head-on rather than hoping the question does not come up
- Appoint internal champions — identify two to three enthusiastic early adopters from the pilot cohort and give them the knowledge and resources to help their peers
- Publish a plain-language FAQ — cover common concerns: does Copilot read my personal emails, can my manager see what I ask Copilot, does Copilot store my queries?
- Build skills alongside the tool — Copilot prompting is a skill; users who receive even a one-hour structured training session generate measurably better results than users who receive no training
What a Successful Pilot Looks Like
At the end of a 90-day pilot, a well-run program produces three things:
- A validated security baseline — you know your Secure Score has improved, your DLP policies are working, your sensitivity labelling coverage has increased, and no data exposure incidents occurred during the pilot
- A clear business case — you have usage data, satisfaction scores, and specific examples of productivity improvement that justify the cost of Copilot licences at scale
- A deployment-ready governance framework — your oversharing remediation, DLP policies, and sensitivity label taxonomy are tested and ready to extend to the full organisation
If your pilot does not produce all three, that is important information. It means either your environment needs more remediation before full deployment, or the Copilot use cases you piloted are not compelling enough to drive adoption at scale. Both outcomes are more valuable than discovering these problems after a full enterprise rollout.
Start with a structured Copilot deployment readiness assessment to understand where your organisation sits before the pilot begins — it will save you significant rework during the pilot itself.
