GitLab Checkmarx CI/CD Integration

calendar Jan 15, 2022 · 9 min read · GitLab Checkmarx  ·
Share on: linkedin copy

Overview

Purpose

Checkmarx is a good tool to do code analysis, but there are 2 pain points during the flow.

  1. Checkmarx can integrate SCM like GitHub/GitLab/..., but it cannot be triggered by by a code commit, it can only schedule the job to run the scan. (If I don't commit code, I don't want to waste resources scanning the code.)
  2. Checkmarx has a built-in issue tracking system, but I think most people prefer to use their issue tracking system in their Git or JIRA or ...

What we want is to integrate Checkmarx scan into GitLab's CI/CD flow including

  1. When we commit code to GitLab, it will trigger Checkmarx to scan automatically.
  2. After the scan, Checkmarx will create issues in GitLab.

Procedure

GitLab admin configuration

  1. You need to set some global variables in GitLab CI/CD admin (Menu->Admin->Settings->CI/CD->Variables). You need to set CHECKMARX_XXX and CX_TEAM variables to integrate Checkmarx. And if you want Checkmarx to create GitLab issue, please also set GITLAB_URL and GITLAB_TOKEN. (You can reference here for the detail)

GitLab project configuration

  1. It's very easy to enable Checkmarx analysis in your GitLab jobs, you just need to edit your .gitlab-ci.yml and include Checkmarx's yml into it (line 1). And you also need to assign Checkmarx's project name in this yml file (line 4).

    1include: 'https://raw.githubusercontent.com/checkmarx-ltd/cx-flow/develop/templates/gitlab/v3/Checkmarx.gitlab-ci.yml'
2
3variables:
4    CX_PROJECT: ProjectXXX   #The project name you want to show in Checkmarx

  2. Then you can try to commit code to trigger a CI/CD flow, Checkmarx analysis will be triggered in 2 conditions:

    1. When you create a merge request in GitLab.
    2. When you commit code to master stream directly.

    The following is a log sample, please check line 20, you can see the variables you defined in GitLab.

      1Running with gitlab-runner 14.5.2 (e91107dd)
  2  on Runner in PC vYY9jCrR
  3Preparing the "docker" executor
  400:06
  5Using Docker executor with image checkmarx/cx-flow ...
  6Pulling docker image checkmarx/cx-flow ...
  7Using docker image sha256:22f48d49aa64275d09b1650c359c0a9b1ab9fa771efa01fba4af89f511b93481 for checkmarx/cx-flow with digest checkmarx/cx-flow@sha256:2b2a2f09680e6c3ba21b00cf8ab4005baa133f0eeeaeb406f09b01a832fffb67 ...
  8Preparing environment
  900:01
 10Running on runner-vyy9jcrr-project-33-concurrent-0 via 435566b8ed7c...
 11Getting source from Git repository
 1200:14
 13Fetching changes...
 14
 15......
 16
 17
 18Executing "step_script" stage of the job script
 19Using docker image sha256:22f48d49aa64275d09b1650c359c0a9b1ab9fa771efa01fba4af89f511b93481 for checkmarx/cx-flow with digest checkmarx/cx-flow@sha256:2b2a2f09680e6c3ba21b00cf8ab4005baa133f0eeeaeb406f09b01a832fffb67 ...
 20$ ${CX_FLOW_EXE} --scan --app="${CI_PROJECT_NAME}" --namespace="${CI_PROJECT_NAMESPACE}" --repo-name="${CI_PROJECT_NAME}" --repo-url="${CI_REPOSITORY_URL}" --cx-team="${CX_TEAM}" --cx-project="${CX_PROJECT}" --branch="${CI_COMMIT_BRANCH}" --spring.profiles.active="${CX_FLOW_ENABLED_VULNERABILITY_SCANNERS}" --f=. ${PARAMS}
 21SLF4J: Class path contains multiple SLF4J bindings.
 22SLF4J: Found binding in [jar:file:/app/cx-flow.jar!/BOOT-INF/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 23SLF4J: Found binding in [jar:file:/app/cx-flow.jar!/BOOT-INF/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
 24SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
 25SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
 26   ___             ___ _
 27  / __\_  __      / __\ | _____      __
 28 / /  \ \/ /____ / _\ | |/ _ \ \ /\ / /
 29/ /___ >  <_____/ /   | | (_) \ V  V /
 30\____//_/\_\    \/    |_|\___/ \_/\_/
 312022-01-21 06:44:09.841  WARN 11 --- [           main] o.s.b.StartupInfoLogger                   [] : InetAddress.getLocalHost().getHostName() took 5006 milliseconds to respond. Please verify your network configuration.
 322022-01-21 06:44:14.858  INFO 11 --- [           main] c.c.f.CxFlowApplication                   [] : Starting CxFlowApplication with PID 11 (/app/cx-flow.jar started by root in /builds/*********)
 332022-01-21 06:44:14.858  INFO 11 --- [           main] c.c.f.CxFlowApplication                   [] : The following profiles are active: sast
 342022-01-21 06:44:18.305  INFO 11 --- [           main] ptablePropertiesBeanFactoryPostProcessor  [] : Post-processing PropertySource instances
 352022-01-21 06:44:18.464  INFO 11 --- [           main] c.u.j.EncryptablePropertySourceConverter  [] : Converting PropertySource configurationProperties [org.springframework.boot.context.properties.source.ConfigurationPropertySourcesPropertySource] to AOP Proxy
 362022-01-21 06:44:18.467  INFO 11 --- [           main] c.u.j.EncryptablePropertySourceConverter  [] : Converting PropertySource commandLineArgs [org.springframework.core.env.SimpleCommandLinePropertySource] to EncryptableEnumerablePropertySourceWrapper
 372022-01-21 06:44:18.467  INFO 11 --- [           main] c.u.j.EncryptablePropertySourceConverter  [] : Converting PropertySource systemProperties [org.springframework.core.env.PropertiesPropertySource] to EncryptableMapPropertySourceWrapper
 382022-01-21 06:44:18.468  INFO 11 --- [           main] c.u.j.EncryptablePropertySourceConverter  [] : Converting PropertySource systemEnvironment [org.springframework.boot.env.SystemEnvironmentPropertySourceEnvironmentPostProcessor$OriginAwareSystemEnvironmentPropertySource] to EncryptableSystemEnvironmentPropertySourceWrapper
 392022-01-21 06:44:18.468  INFO 11 --- [           main] c.u.j.EncryptablePropertySourceConverter  [] : Converting PropertySource random [org.springframework.boot.env.RandomValuePropertySource] to EncryptablePropertySourceWrapper
 402022-01-21 06:44:18.469  INFO 11 --- [           main] c.u.j.EncryptablePropertySourceConverter  [] : Converting PropertySource applicationConfig: [classpath:/application-sast.yml] [org.springframework.boot.env.OriginTrackedMapPropertySource] to EncryptableMapPropertySourceWrapper
 412022-01-21 06:44:18.469  INFO 11 --- [           main] c.u.j.EncryptablePropertySourceConverter  [] : Converting PropertySource applicationConfig: [classpath:/application.yml] [org.springframework.boot.env.OriginTrackedMapPropertySource] to EncryptableMapPropertySourceWrapper
 422022-01-21 06:44:18.607  INFO 11 --- [           main] c.u.j.f.DefaultLazyPropertyFilter         [] : Property Filter custom Bean not found with name 'encryptablePropertyFilter'. Initializing Default Property Filter
 432022-01-21 06:44:18.620  INFO 11 --- [           main] c.u.j.r.DefaultLazyPropertyResolver       [] : Property Resolver custom Bean not found with name 'encryptablePropertyResolver'. Initializing Default Property Resolver
 442022-01-21 06:44:18.629  INFO 11 --- [           main] c.u.j.d.DefaultLazyPropertyDetector       [] : Property Detector custom Bean not found with name 'encryptablePropertyDetector'. Initializing Default Property Detector
 452022-01-21 06:44:18.934  WARN 11 --- [           main] j.p.spi                                   [] : javax.persistence.spi::No valid providers found.
 462022-01-21 06:44:18.938  INFO 11 --- [           main] trationDelegate$BeanPostProcessorChecker  [] : Bean 'org.hibernate.validator.internal.constraintvalidators.bv.NotBlankValidator' of type [org.hibernate.validator.internal.constraintvalidators.bv.NotBlankValidator] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
 472022-01-21 06:44:18.942  INFO 11 --- [           main] trationDelegate$BeanPostProcessorChecker  [] : Bean 'org.hibernate.validator.internal.constraintvalidators.bv.NotNullValidator' of type [org.hibernate.validator.internal.constraintvalidators.bv.NotNullValidator] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
 482022-01-21 06:44:18.944  INFO 11 --- [           main] trationDelegate$BeanPostProcessorChecker  [] : Bean 'flowProperties' of type [com.checkmarx.flow.config.FlowProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
 492022-01-21 06:44:18.947  INFO 11 --- [           main] trationDelegate$BeanPostProcessorChecker  [] : Bean 'flowAsyncConfig' of type [com.checkmarx.flow.config.FlowAsyncConfig$$EnhancerBySpringCGLIB$$9e6d5344] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
 502022-01-21 06:44:22.959  INFO 11 --- [           main] o.s.w.s.s.SaajSoapMessageFactory          [] : Creating SAAJ 1.3 MessageFactory with SOAP 1.1 Protocol
 512022-01-21 06:44:23.569  INFO 11 --- [           main] o.s.s.c.ThreadPoolTaskExecutor            [] : Initializing ExecutorService
 522022-01-21 06:44:23.571  INFO 11 --- [           main] o.s.s.c.ThreadPoolTaskExecutor            [] : Initializing ExecutorService 'scanRequest'
 532022-01-21 06:44:23.576  INFO 11 --- [           main] o.s.s.c.ThreadPoolTaskExecutor            [] : Initializing ExecutorService
 542022-01-21 06:44:23.577  INFO 11 --- [           main] o.s.s.c.ThreadPoolTaskExecutor            [] : Initializing ExecutorService 'webHook'
 552022-01-21 06:44:23.790  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [] : =======BUILD INFO=======
 562022-01-21 06:44:23.791  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [] : Version: cx-flow-1.6.29
 572022-01-21 06:44:23.792  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [] : Time: 2022-01-07T12:45:49.239Z
 582022-01-21 06:44:23.793  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [] : =======================
 592022-01-21 06:44:25.426  INFO 11 --- [           main] c.c.f.CxFlowApplication                   [] : Started CxFlowApplication in 32.909 seconds (JVM running for 34.89)
 602022-01-21 06:44:25.457  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [blKXd4Ey] : Using custom bean implementation  for bug tracking
 612022-01-21 06:44:25.539  INFO 11 --- [           main] c.c.f.s.ScaFilterFactory                  [blKXd4Ey] : Initializing SCA filters.
 622022-01-21 06:44:25.541  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [blKXd4Ey] : Executing scan process
 632022-01-21 06:44:25.551  INFO 11 --- [           main] c.c.f.s.ScaFilterFactory                  [blKXd4Ey] : Initializing SCA filters.
 642022-01-21 06:44:25.571  INFO 11 --- [           main] c.c.f.s.ProjectNameGenerator              [blKXd4Ey] : Project name being used: *********
 652022-01-21 06:44:25.573  INFO 11 --- [           main] c.c.f.u.ZipUtils                          [blKXd4Ey] : Creating zip file /builds/*********/cx.1357f707-5066-486c-b6f5-1d9dc08a4ad9.zip from contents of path .
 662022-01-21 06:44:25.573  INFO 11 --- [           main] c.c.f.u.ZipUtils                          [blKXd4Ey] : Applying exclusions: .jar
 672022-01-21 06:44:47.686  INFO 11 --- [           main] c.c.f.u.ZipUtils                          [blKXd4Ey] : Successfully created /builds/*********/cx.1357f707-5066-486c-b6f5-1d9dc08a4ad9.zip 
 682022-01-21 06:44:47.688  INFO 11 --- [           main] c.c.f.s.ScanRequestConverter              [blKXd4Ey] : Overriding team with /CxServer/TEAM***
 692022-01-21 06:44:47.702  INFO 11 --- [           main] c.c.s.s.CxAuthService                     [blKXd4Ey] : Logging into Checkmarx https://***.com/cxrestapi/auth/identity/connect/token
 702022-01-21 06:44:48.665  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Retrieving Cx teams
 712022-01-21 06:44:48.700  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Found team /CxServer/TEAM*** with ID 16
 722022-01-21 06:44:48.920  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Creating scan...
 732022-01-21 06:44:48.921  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Updating Source details for project Id 67
 742022-01-21 06:45:05.838  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Scan will be Full Scan
 752022-01-21 06:45:05.840  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Creating Scan for project Id 67
 762022-01-21 06:45:06.021  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Scan created with Id 1000207 for project Id 67
 772022-01-21 06:45:06.171  INFO 11 --- [           main] jsonLogger                                [blKXd4Ey] : 
 782022-01-21 07:22:12.549  INFO 11 --- [           main] jsonLogger                                [blKXd4Ey] : 
 792022-01-21 07:22:12.551  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Creating report for xml Id 1000207
 802022-01-21 07:22:12.906  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Report with Id 304 created
 812022-01-21 07:22:17.908  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Retrieving report status of report Id 304
 822022-01-21 07:22:18.965  INFO 11 --- [           main] c.c.s.s.CxAuthService                     [blKXd4Ey] : Logging into Checkmarx for SOAP token https://***.com/cxrestapi/auth/identity/connect/token
 832022-01-21 07:22:19.117  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Retrieving report contents of report Id 304 in XML format
 842022-01-21 07:22:19.220  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Report downloaded for report Id 304
 852022-01-21 07:22:19.805  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Fetching Scan data for Id 1000207
 862022-01-21 07:22:19.820  INFO 11 --- [           main] c.c.s.s.CxService                         [blKXd4Ey] : Fetching custom fields from project ID 67
 872022-01-21 07:22:19.908  INFO 11 --- [           main] c.c.f.s.ResultsService                    [blKXd4Ey] : Issue tracking is custom bean implementation
 882022-01-21 07:22:19.923  INFO 11 --- [           main] c.c.f.c.GitLabIssueTracker                [blKXd4Ey] : Initializing GitLab processing
 892022-01-21 07:22:20.325  INFO 11 --- [           main] c.c.f.s.CodeBashingService                [blKXd4Ey] : not using CodeBashing lessons integration - one or more of the mandatory properties is missing
 902022-01-21 07:22:20.325  INFO 11 --- [           main] c.c.f.s.IssueService                      [blKXd4Ey] : Processing Issues with custom bean GitLab
 912022-01-21 07:22:20.325  INFO 11 --- [           main] c.c.f.c.GitLabIssueTracker                [blKXd4Ey] : Executing getIssues GitLab API call
 922022-01-21 07:22:20.465  INFO 11 --- [           main] c.c.f.s.IssueService                      [blKXd4Ey] : Issue still exists.  Updating issue with key CX Second_Order_SQL_Injection @ ***.cs [master]
 932022-01-21 07:22:21.352  INFO 11 --- [           main] c.c.f.c.GitLabIssueTracker                [blKXd4Ey] : Finalizing GitLab Processing
 942022-01-21 07:22:21.352  INFO 11 --- [           main] c.c.f.s.ResultsService                    [blKXd4Ey] : ####Checkmarx Scan Results Summary####
 952022-01-21 07:22:21.352  INFO 11 --- [           main] c.c.f.s.ResultsService                    [blKXd4Ey] : Team: /CxServer/TEAM***, Project: *********, Scan-Id: 1000207
 962022-01-21 07:22:21.353  INFO 11 --- [           main] c.c.f.s.ResultsService                    [blKXd4Ey] : The vulnerabilities found for the scan are: high: 2, medium: 21, low: 156, info: 0
 972022-01-21 07:22:21.353  INFO 11 --- [           main] c.c.f.s.ResultsService                    [blKXd4Ey] : To view results use following link: https://***.com/CxWebClient/ViewerMain.aspx?scanid=1000207&projectid=67
 982022-01-21 07:22:21.353  INFO 11 --- [           main] c.c.f.s.ResultsService                    [blKXd4Ey] : ######################################
 992022-01-21 07:22:21.353  INFO 11 --- [           main] c.c.f.s.ThresholdValidatorImpl            [blKXd4Ey] : Checking Thresholds exists. sast thresholds: false. sca thresholds: false
1002022-01-21 07:22:21.353  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [blKXd4Ey] : Build succeeded. all checks passed
1012022-01-21 07:22:21.353  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [blKXd4Ey] : Completed Successfully
1022022-01-21 07:22:21.354  INFO 11 --- [           main] c.c.f.CxFlowRunner                        [blKXd4Ey] : Finished with exit code: 0
1032022-01-21 07:22:21.359  INFO 11 --- [extShutdownHook] o.s.s.c.ThreadPoolTaskExecutor            [] : Shutting down ExecutorService 'webHook'
1042022-01-21 07:22:21.362  INFO 11 --- [extShutdownHook] o.s.s.c.ThreadPoolTaskExecutor            [] : Shutting down ExecutorService 'scanRequest'
105Cleaning up project directory and file based variables
10600:05
107Job succeeded

  3. During the Checkmarx analysis, you can check the status from its GUI, it takes a long time to analyze.

  4. After the analysis, you can check the result in Checkmarx or GitLab.

    1. Check the result in Checkmarx.

    2. Check the result in GitLab, it generates 1 report in merge request and it will create several issues by category. The following is the reqport of a merge request. In this report, there are 9 high issues (in 5 categories).

      1. The following are the 5 issues created by Checkmarx in GitLab.

      2. You can click above issue, it shows the detail vulnerability information.

Troubleshooting

  1. If you find Checkmarx cannot find your team, please make sure to use English only in CX_TEAM, if you use non-English (ex: Chinese or Japanese), you can check the log and find it cannot find the team. 
    12022-01-12 02:51:22.748  INFO 11 --- [main] c.c.s.s.CxService [x4DNMhOL] : Found team /CxServer/Team1 with ID 16

Reference

  1. You can reference the official document to do the integration: https://checkmarx.atlassian.net/wiki/spaces/SD/pages/1929937052/GitLab+Integration.

Conclusion

  1. After this integration, you just need to commit the code and see the result in GitLab. All flows are in GitLab now, you don't need to worry about Checkmarx.
  2. Checkmarx provides lots of integration, you can reference https://checkmarx.atlassian.net/wiki/spaces/SD/pages/1339162785/CxSAST+CxSCA+Plugins+and+Integrations.

What's next

  1. Try to integrate the scan result to SonarQube), you can reference https://dennys.github.io/en/doc/devops/gitlab-checkmarx-sonarqube-integration/.

Posts in this Series