Thursday, April 21, 2016

Lessons Learned from Infiltrate Training 2016

Every year I write up a lessons learned blog post that gives some insight into what goes on behind the scenes when producing our security training. Since all of the Infiltrate training happened at the same time this year I'll provide a few lessons from other classes but the majority will from our WebHacking course.

Overview

We fielded four classes this year:
Master Class - Max, Rod, Matias
Click Here for Ring 0 - Facundo, Lurene
Wide Open to Interpretation - Esteban, Enrique
Web Hacking - AlexM, Miguel, David

Hardware matters, a lot

We provide hardware to students for our classes, this year we switched to Lenovo z50s. Despite running the latest XUbuntu their ability to interface with a projector over HDMI was absolute garbage and a horrible headache. Also, they use an AMD processor rather than an Intel one. Some of our Linux kernel master class content relied on Intel specific tricks. We had a surprisingly high number of those laptops die due to bad RAM in the Click Here course.

Lesson: Make sure the lead instructors for any class you teach sign off on hardware changes

Think about distribution

We stopped printing physical copies of the slides for students in 2016 and instead gave each student their own PDFs. In Web Hacking that wasn't hard to do because we've got a scoring server where students have individual accounts and can submit tokens for competition. Other classes had an issue because there wasn't a scoring server concept.

Lesson: If you're giving out slides just provide students with a pre-loaded USB stick at the beginning of class. I still always recommend each class have a utility web server on the internal network for distributing files during class as needed.


Obvious bugs can have long lives

Despite running the scoring server for years we only this year discovered that the username field for the login page was case sensitive. This rattled us a bit at the beginning of class as we thought we had some kind of wide-spread login problem with the scoring server.

Lesson: Mentally prepare yourself for the reality that your students will find bugs that you missed


Scoring can be helpful

I've written a lot in the past about some of the benefits and drawbacks of including a competitive component in your class. One of the key things we use it for in WebHacking is to get a sense where individual students may be struggling or which exercises need to be tweaked because only a few people have solved them. When I asked other trainers how students in their classes were doing they had a much looser sense of performance where as we had pretty solid data.

Lesson: Create concrete metrics you can obtain throughout the class to determine how students are doing other than self reporting


Scoring can be harmful

Not all students want to participate in the competitive aspect of the class. We make it very clear at the beginning that it is entirely voluntary. But in practice we erred on the side of not going into solutions for some problems because certain students were still competing. This was clearly a disservice on our part as we could have done more to ensure students understood some of the trickier content.

Lesson: Don't let the competitive aspect of the class negatively impact the learning of many students


Students want homework

Our number one request this year was to provide access to exercises outside of class. We've struggled with that a little bit because it meant changing our architecture and providing some kind of support after hours. We still haven't entirely decided how we're going to solve this request.

Lesson: Create your environment to provide remote access from the get-go so students can continue learning after class hours

 

Students expect all solutions

We have a TON of content in our Web Hacking class, right now the class is four days and could easily stretch to five if we went over each and every solution manually. Rather than doing that we're going to be producing solution PDFs that provide in depth discussions and solutions for each exercise. This gives the students the ability to review solutions after class hours and come back in the morning with questions.

Lesson: Any time you create a new exercise or task for students you should write up a complete solution with corresponding documentation explaining how they would figure out each step