While building “Husky AI” I started working a lot with Microsoft’s VS Code Python extension. It is a super convinient way to edit Jupyter Notebooks. I just use VS Code’s Remote SSH feature to get to my Linux host and work on modeling and testing there.

When threat modeling “Husky AI” I identified backdooring of third party libraries and development tools as a potential issue to be aware of.

So finding security issues in the tools is naturally something I am keeping an eye on.

HTML and JavaScript Injection via traceback

While using VS Code and editing Notebooks I noticed that it was vulnerable to HTML and JavaScript injections under certain conditions. Which means that a Notebook could be backdoored, stealing the user’s information and worst case possibly take over the victims computer.

Here is the screenshot I sent over to MSRC as part of the bug report.

The attack could also be done by modifying a notebook file directly and adding malicous code to it.

Additionally, the vulnerability could also be triggered remotely in case the Notebook connects to a remote server where a sneaky attacker controls information that cases an error to be rendered in the Notebook - the vulnerability was in the error/exception scenario (referred to as traceback) where strings were not correctly escaped.

This was the kind of threat that I was thinking of during threat modeling.

I reported this to MSRC and they confirmed the vulnerability with a severity rating important and the team fixed it. The bulletin is published here. See the timeline below for details.

Disclosure timeline

• August, 25th 2020 - Informing MSRC about the vulnerability
• August, 26th 2020 - MSRC replies and a case was opened
• September, 8th 2020 - Pinging MSRC to check on triage
• September, 14th 2020 - Pinging MSRC to check on triage
• September, 15th 2020 - MSRC confirmed. Working on fix and CVE assigned
• September, 28th 2020 - Update from MSRC, fixed and will be disclosed in October
• October, 14th 2020 - Case closed

GrayHat - Red Team Village talk: Building and breaking a machine learning system

Make sure to swing by my talk at the Red Team Village at GrayHat October, 31st.

I will be talking about more interesting machine learning attacks in my talk “Learning by doing: Building and breaking a machine learning system”.

Hope to see many of you.

Cheers, Johann.