While building “Husky AI” I started working a lot with Microsoft’s VS Code Python extension. It is a super convinient way to edit Jupyter Notebooks. I just use VS Code’s Remote SSH feature to get to my Linux host and work on modeling and testing there.
When threat modeling “Husky AI” I identified backdooring of third party libraries and development tools as a potential issue to be aware of.
So finding security issues in the tools is naturally something I am keeping an eye on.
Here is the screenshot I sent over to MSRC as part of the bug report.
The attack could also be done by modifying a notebook file directly and adding malicous code to it.
Additionally, the vulnerability could also be triggered remotely in case the Notebook connects to a remote server where a sneaky attacker controls information that cases an error to be rendered in the Notebook - the vulnerability was in the error/exception scenario (referred to as traceback) where strings were not correctly escaped.
This was the kind of threat that I was thinking of during threat modeling.
I reported this to MSRC and they confirmed the vulnerability with a severity rating important and the team fixed it. The bulletin is published here. See the timeline below for details.
- August, 25th 2020 - Informing MSRC about the vulnerability
- August, 26th 2020 - MSRC replies and a case was opened
- September, 8th 2020 - Pinging MSRC to check on triage
- September, 14th 2020 - Pinging MSRC to check on triage
- September, 15th 2020 - MSRC confirmed. Working on fix and CVE assigned
- September, 28th 2020 - Update from MSRC, fixed and will be disclosed in October
- October, 14th 2020 - Case closed
GrayHat - Red Team Village talk: Building and breaking a machine learning system
Make sure to swing by my talk at the Red Team Village at GrayHat October, 31st.
I will be talking about more interesting machine learning attacks in my talk “Learning by doing: Building and breaking a machine learning system”.
Hope to see many of you.