At AppSecUSA, OWASP Glue, a project we contribute heavily to, was asked to present in the project showcase. I put together an overview talk about how the tool is architected and what we use it for. Then, we added a new task live during the talk. I thought that was enough fun that it was worth blogging about. I would like to start by acknowledging Omer Levi Hevroni, of Soluto, who has also contributed and continued support for Glue over the past year +.
We started working on Glue because we found that most security tools don’t integrate all that cleanly with CI/CD or other developer oriented systems. Our goal is always to get the checking as close to the time of code authorship as possible. Further, we thought that having it wrap open source tools would make them more accessible. Philosophically, I’m not that excited about tools (see earlier post) but we should definitely be using them as much as we can provided they don’t get in our way. Ultimately, we wrote Glue to make it easy to run a few security tools and have the output dump into Jira or a CSV.
So we defined Glue with the following core concepts:
- Mounters – these make the code available. Could be pulling from GitHub or opening a docker image.
- Tasks: Files – these analyze files. Examples are ClamAV and hashdeep.
- Tasks: Code – these are some level of code analysis. Some are first class static tools, others are searching for dependencies or secrets.
- Tasks: Live – these are running some kind of live tool like Zap against a system.
- Filters – filters take the list of findings and eliminate some based on whatever the filter criteria are. An example is limiting ZAP X-Frame-Options findings to 1 per scan instead of 1 per page.
- Reporters – reporters take the list of findings and push them wherever you want them to end up. JIRA, TeamCity, Pivotal Tracker, CSV, JSON, etc.
OK, so a common task is to say
“Hey, this is cool but I want to do this with a tool that Glue doesn’t yet support.”
Great. That should be easy. Let’s walk through the steps. We’ll do this with a python dependency checker: safety.
Set Up the Task
A good way to start a new task is to copy an existing one. In this case, we’re going to do a python based task so let’s copy the bandit task and alter to match:
Now we have the start of a class that defines our SafetyCheck task.
Run the Tool: Safety Check
Here we just need to implement the run method and tell the task how to call the tool we want to run. In this case, we want it to create json. The resulting code is:
The @trigger is set when the Task is initialized (see above) and includes information about where the code is. We use that to know where the path that we want to analyze is. Then we use one of the provided util methods runsystem to invoke safety check with the parameters we want.
Note that we are putting the result in the @result instance variable.
Parse the Results
Once the tool runs, we have the output in our @result variable. So we can look at it and parse out the JSON as follows:
Here, we call a clean_result method on the result first. You can look here for detail, but it is just pulling the warnings that the tool emits that make the output not valid JSON. We do this to make sure the JSON is parseable for our output. This is a common issue with open source tools. I don’t underestimate the value of making these things just work.
The magic here is really in the two lines that set the source and then report it. The source in Glue terms is the thing that found the issue and where it found it. In this case, we’re setting it to be our Task (SafetyCheck) and the library name in the output from file containing the dependencies. (requirements.txt)
The report method is supplied by the parent BaseTask class and takes as arguments: description, detail, source, severity and fingerprint.
You can see if you look closely that we set all of the severities to Medium here because safety check doesn’t give us better information. We also use the supplied fingerprint method to make sure that we know if we have a duplicate. You can also see that the result of calling report is that we have a new finding and the finding is added to the array of findings that were created by this task. We get the trigger name, the check name, a timestamp, etc. just by using the common Finding and BaseTask classes.
Making Sure the Tool is Available
In addition to run and analyze the other method we expect to have in our Task is a method called supported?. The purpose of this method is to check that the tool is available. Here’s the implementation we came up with for safety check, which doesn’t have a version call from the CLI.
The idea here is to run the tool in a way that tests if it is available and alerts the user if it is not. Graceful degredation as it were…
Here is the code from the Tasks ruby file that runs all the tasks.
Here you see the supported?, run and analyze methods getting called. You also see the labels and tasks being applied. It’s not magic, but it might look weird when you first jump in.
We wanted to create a new task. We did it in about 20 minutes in a talk. Of course, I wrote the code around it so it was easy. But it does illustrate how simple it is to add a new task to Glue.
If you want to see this check in action, you can run something like:
We’re getting the Docker image updated, but once available you can run:
We hope people can digest and understand Glue. We think it is an easy way to get a lot of automation running quickly. We even run a cluster of Kubernetes nodes doing analysis. This in addition to integrating into Jenkins builds or git hooks. Happy Open Source coding!!!