Here’s my little tale about MDMs. It’s a history, plus how we migrated from one platform to another, why we did, and the ouches along the way.
AirWatch on iPads
At my first MacAdmins at PSU, I was speaking to a fellow macadmin about the pain of managing iPads using Configurator, Apple’s in-house product to manage iPads. He said he was in a similar pain and then moved to AirWatch, a Mobile Device Management system (MDM).
AirWatch, along with Apple’s Device Enrollment Program (DEP) got me where I needed. I could wirelessly provision my iPads and install applications from Apple’s Volume Purchasing Program (VPP), which is basically the App Store for organizations.
macOS in WorkspaceONE
We eventually put our macOS devices into AirWatch, which was now retitled WorkspaceONE, with a very simple workflow. Computer would boot, DEP would tell the computer it was owned by the school and was assigned to WorkspaceONE, WorkspaceONE would install a package with Munki and run a script to rename the computer based on a Google Sheet (that script can be found here).
WorkspaceONE would also be used to install configuration profiles on the Mac for things that an MDM was needed for and couldn’t be done via Munki such as Privacy Preferences Policy Control (PPPC) which requires User Approved MDM (UAMDM) to be deployed.
iPad *headdesk*
All was good, and then stuff didn’t work as well. Apps wouldn’t push out to the devices, configuration profiles wouldn’t push out to the iPads. The Macs were depending on WorkspaceONE for so little that it didn’t really matter. I was helping my friend move away from DeployStudio for his imaging needs and move to no-imaging, I suggested he use Mosyle for his Macs. I liked what I saw and I was tempted.
Since Mosyle was free for one platform (iOS/iPadOS or macOS), in August 2019, I decided to move all my iPads over to Mosyle. It would be easy. I annually wipe all my iPads. Move them over to in Apple School Manager from WorkspaceONE to Mosyle, set up configuration settings, move my VPP licences overs, wipe the iPads and watch them all enroll. It went amazingly.
macOS *headdesk*
We mostly used WorkspaceONE on the macs just to install Munki, but there were a few things it wasn’t doing properly. We setup a firmware password to prevent students from restarting computers into Recovery and changing teacher passwords. It was only successfully installed on 10% of devices. We sent out PPPC settings for Smart Notebook and it only installed for about 80% of fleet. We sent out a kext allowlist which only worked on about 50% of the fleet.
Whenever we called VMWare support, we usually got a support agent who didn’t know the macOS platform. It would take over 24 hours before VMWare would call us. They would always call outside of our normal business hours and any resolution to our problems was in spite of their support staff, not because of them.
My plan was to move all macOS devices over to Mosyle in September 2020. It would be much harder. I can’t just wipe teacher laptops. While there’s no policy in favour of this, many teachers use their school devices as their personal devices. In addition, many don’t store all their data in Google Drive as they’ve been instructed to do for many years. As such, I also pushed back our planned roll out of Catalina until September 2020. Normally I tried to allow teachers to install a new OS via Munki as soon as possible (after testing).
The Best Plans Are Destroyed By A Pandemic
With remote learning, and a closed building, we were managing computers via Zoom. This is fine if WorkspaceONE was pushing out the PPPC policies correctly to allow for the fleet. Our users are Standard users (not Admin, aka, non-privileged users), and thus they cannot authorize the PPPC settings for Accessibility to allow remote control of their computer via Zoom.
Then Apple rolled out a security update that caused major problems in macOS 10.14.6 and Zoom. We had crashes. Terrible crashes. Many were not able to teach.
To Mosyle and Beyond!
Mosyle were kind enough to offer us free usage until the end of June if I signed the full one year contract we were planning to buy next year (July 2020-July 2021). I jumped on that.
I was testing Mosyle for macOS in September, so back then I put all policies and configuration profiles from WorkspaceONE into Mosyle. I needed to do some updating of policies that changed since September. I did that, then I tested on a couple of machines. Then I wiped them, enrolled them in WorkspaceONE and tested the migration process to Mosyle. All seemed to go well.
Then I logged into the computer lab at the school. I tested the migration process on those computers, it went simply and quickly. Then I remembered that I don’t have Remote Desktop access to the computers at teacher homes. I’m running this through Zoom and a Standard User. So with a bit of a chat with Rich Trouton of Der Flounder fame, I confirmed that his software Privileges, if deployed through Munki, would give the user elevated privileges and allow me to walk them through the final process.
The Process
- First day, distribute to all devices via Munki a stub installer of Catalina
I used the stub rather than the full installer because the download from Apple’s servers would be faster than the download from the school’s server - Day before, add Privileges to the computer’s manifest in Munki as a Managed Install
- Switch computer from WorkspaceONE to Mosyle in Apple School Manager
- Check on WorkspaceONE if Privileges had been installed, if so, choose “Delete Device”
- Connect via Zoom, and have the user share their Desktop
- Request control, they would get a message asking either to open System Preferences to allow or Deny
- Ask the user to open System Preferences
- Ask the user to launch Privileges from the Applications folder and request privileges
- Have the user allow Accessibility for Zoom in the Privacy pane of System Preferences
- Take control and use Privileges to revoke privileges
- Put the Privileges app in the Managed Uninstalls for the device’s manifest in Munki
- Confirm the profiles are removed from the computer and it is unenrolled from WorkspaceONE
- Go to enroll.mosyle.com/?account=school and download the profile to enroll in Mosyle
- Assign the device in Mosyle to the appropriate user (teacher-only and admin-only profiles will be pushed depending on who it is assigned to)
- Run Managed Software Centre to remove Privileges
- Run the Catalina stub and tell the device to install 10.15.4
While in my testing everything worked like a charm, that didn’t translate to the real world.
What Went Wrong
For about 75% of the computers everything went perfectly. For 5% of the computer, it could take anywhere from an hour to 24 hours to delete the device from WorkspaceONE. Sometimes a reboot triggered it, sometimes a it just happened when it felt like it.
After days of trying to get help from VMWare, I was finally told by the MacAdmins slack that Delete Device is not the best way to do this. What I wanted was Enterprise Wipe, which removes all traces of the MDM (in theory). To me using the word “wipe” had some bad connotations and scared me away from using it.
I tested the Enterprise Wipe function on the computer lab iMacs and it worked like a charm. It could still take anywhere from 1 hour to 24 hours, but at least the support agent that was assigned to help me1 gave me a bit less grief if I used Enterprise Wipe rather than Delete Device.
There was still the remaining 20% of devices. There seemed to be a theme between those 20%. They all had enrolled in WorkspaceONE in September 2019 and never communicated with the system again.
I was installing the WorkspaceONE agent, on the computer to get it to reestablish communications with the MDM, and that worked, but once you told it to perform an enterprise wipe, it wouldn’t wipe.
Days would go by and no Enterprise Wipe.
I Guess We’re Disabling SIP? (Temporarily)
In the end I was kinda forced to do this. I didn’t want to, but I kinda had to.
- Connect with the user over the phone (there will be numerous restarts so Zoom won’t work)
- Diable SIP
- Have them restart the computer holding down Command-R
- Click Utilities
- Click Terminal
- Type
csrutil disable
and return2 - Restart the computer (Apple menu, restart)
- Connect via Zoom, keep the mic off as you’re still on the phone with them, screen share and request control
- Launch the terminal
su <<Admin User Name here>>
sudo rm -rf /var/db/ConfigurationProfiles/
sudo rm /Library/Keychains/apsd.keychain
sudo reboot
- Connect via Zoom, keep the mic off as you’re still on the phone with them, screen share and request control
- Go into System Preferences and make sure there are no profiles
- Enable SIP (has to be done before enrolling in Mosyle, because Mosyle will actually install the firmware password profile)
- Have them restart the computer holding down Command-R
- Click Utilities
- Click Terminal
- Type
csrutil enable
and return3 - Restart the computer (Apple menu, restart)
- Connect via Zoom, keep the mic off as you’re still on the phone with them, screen share and request control
- Enroll in Mosyle
- Go to enroll.mosyle.com/?account=school and download the profile to enroll in Mosyle
- Assign the device in Mosyle to the appropriate user (teacher-only and admin-only profiles will be pushed depending on who it is assigned to)
So, that was my tale. I hope it helps someone. I hope that someone at VMWare sees this and tries to figure out why their support is so bad.
- I don’t want to use any verbiage to imply that she did actually help me, because she didn’t. I don’t even want to suggest she tried to help me, because she didn’t. [↩]
- Text this to them, so you don’t have to spell that out over the phone. [↩]
- Text this to them, so you don’t have to spell that out over the phone. [↩]
Pingback: Missing Profiles Button in System Preferences | Never Had To Fight