HOW MATURE
IS YOUR CERT?

FAQ CERTrating EN

What is CERTrating

CERTrating is the first online platform to evaluate the maturity level of CERTs and their services.

Why use CERTrating?

With CERTrating you can measure the maturity level of your CERT also in relation to the maturity of each individual service and the role they play in achieving corporate goals.
It helps you identify the improvement strategies best suited to your needs and provides you an overall view of the CERT status for the Management and its ranking with other CERTs.

What is SIM3?

SIM3 is the acronym of "Security Incident Management Maturity Model". It is a model adopted by ENISA for the maturity level evaluation of the CSIRT / CERT. It is composed of questionnaire of 44 multiple choice questions that evaluates maturity on three different levels: BASIC, INTERMEDIATE, ADVANCED.
The logic applied by CERTrating is based truthfully as the same adopted by ENISA SIM3 mode.

Why CERTrating is different from the ENISA model?

ENISA model evaluates only the overall level of the CERT but not for the services provided. CERTrating allows you to evaluate the maturity level of CERT’s services and a "custom" maturity of the entire CERT, which in this case, takes into account also the maturity and the importance attributed to services.
CERTrating also suggests the minimum and optimal actions that should be implemented to raise the next maturity level for the CERT and its services, providing a dashboard and reports for the Management.

Does CERTrating also consider the NIS Directive?

Yes, ENISA published in April 2019 the updated model that considers the NIS directive requirements.

What are the needs pursued by the Maturity Model for CERTs today?

ENISA defines the needs and aims pursued for all CERTs:
"Today the role and functions of the national and governmental CSIRTs of the countries are evolving and therefore the teams that must keep pace with the demands and expectations are growing. The improvement of maturity allows the teams to constantly improve their skills".
Cit. CSIRT Capabilities, How to assess maturity?, Guidelines for national and governmental CSIRTs, DECEMBER 2015, https://www.enisa.europa.eu/publications/csirt-capabilities

How does CERTrating evaluate CERTs maturity level?

CERTs Maturity level is calculated according to the logic implemented by ENISA using the matrix that summarizes the three maturity levels (BASIC, INTERMEDIATE, ADVANCED) and the individual requirements for each parameter (10 Organization, 7 Human, 10 Tool, 17 Process parameters). (Annex "A" ENISA CSIRT maturity assessment model https://www.enisa.europa.eu/publications/study-on-csirt-maturity).
The Criteria and levels of Maturity adopted by ENISA and applied also by CERTrating are:

1. The thematic subdivision of Self-Assessment in 4 quadrants or parameters divided into: Organization, Resources, Tools and Processes
2. An increasing Maturity of Answers given to questions (Levels from 0-4)
3. Overall Maturity Level of the CSIRT / CERT (Not Basic, Basic, Intermediate, Advanced)
4. The matrix of correspondence between the level of overall maturity and the level of maturity of the individual responses

What is a Maturity Model?

Capability Maturity Model (CMM) is a model for assessing the maturity of processes in a company as a whole, from the technology used, to the organization, staff and training. Methods, procedures, tools and equipment are considered too.
ENISA has decided to apply this model, in particular the SIM3 model, to CSIRT / CERT in order to help them to improve their maturity.

What is the maturity model applied to the Self-Assessment answer?

The maturity model applied for the levels is valid for all the Parameters in all Quadrants of the Self-Assessment both of the CERT and of the CERT Services and can be summarized according to the following criteria:

• 0 = not available / undefined / unaware
• 1 = implicit (known/considered but not written down, "between the ears")
• 2 = explicit, internal (written down but not formalized in any way)
• 3 = explicit, formalized on authority of CSIRT head (rubberstamped or published)
• 4 = explicit, audited on authority of governance levels above the CSIRT head (subject to control process/audit/enforcement)

What do the Basic, Intermediate, Advanced levels mean?

The Basic, Middle, Advanced tripartition has been defined by ENISA and represents the overall maturity level resulting from the Self-Assessment surveys conclusion.

• Basic Maturity Level: For this level, activities on all parameters have been started with a clear focus on the mandate and other formal considerations of the team's role. Approximately 80% of the organizational parameters have already been addressed to such a degree, that they can be considered "advanced".
• Intermediate Maturity Level: Based on the work done so far, progress for all parameters, except for those already on "advanced" level, has been achieved. Overall, approximately 50% of the human, tool and process parameters can be considered "advanced".
• Advanced Maturity Level: The final step directs the efforts to the remaining parameters and achieves a level that is considered "advanced".

I have completed the self-assessment survey answering to all the questions but my maturity level is "Not Basic" why?

In order to be able to reach the overall maturity level of the CERT, whether Basic, Intermediate or Advanced, ALL the answers provided HAVE TO REACH EXACTLY the minimum level defined for BASIC, INTERMEDIATE, ADVANCED defined by ENISA. If EVEN ONE of the answers provided does not reach one of the three defined levels, the overall maturity level of the CERT will be non-basic (or the previous level).

The reference matrix is available in the "Annex A" of the ENISA CSIRT maturity assessment model https://www.enisa.europa.eu/publications/study-on-csirt-maturity).

Could I have an example of how the matrix criteria works?

Certainly. I have answered all the questions and the system gives me the Final Maturity level: "Not Basic". Why?

For example, if in 43 questions the user has respected or exceeded in the answers the level defined by ENISA for "BASIC" Maturity level and ONLY 1 question does not reach the minimum measure defined by ENISA for that level, the system will evaluate the level "Not Basic".

How are services maturity calculated?

The services maturity level is evaluated respecting the model and criteria and the matrix defined by ENISA (Annex "A" ENISA CSIRT maturity assessment model https://www.enisa.europa.eu/publications/study-on-csirt-maturity).
The services’ self-assessments surveys respect the same parameters for the evaluation of CERTs maturity but questions and answers have been customized for each service.

What "CERT Custom maturity level" is?

The "CERT Custom maturity level" is one of the improvements introduced by CERTrating. This overall-CERT-level considers the maturity level and the weight attributed to the services. If weights have not been set, the system automatically assigns the same weight to all services.

How are the services identified?

ENISA has identified the following CSIRT / CERT the services, divided up as follows:

• Reactive Services: Alerts & Warnings, Incident handling, Vulnerability handling, Artifact handling
• Proactive Services: Announcements, Tech Watch, Security Audits / Pentests, Tools development, Intrusion Detection, Threat intelligence sharing
• Security Quality Management services: Risk Analysis, BC & DR planning, Security awareness, Training

What are services weights used for?

CERTrating offers CERTs the possibility to select CERT own services and assign them a weight that varies from 1 to 4. This value will contribute to determine the CERT Custom maturity.

What can I see on the Dashboard?

The Dashboard offers an instant view for the Management of the maturity level of the CERT, its services and its ranking. The ranking considers both maturity level of CERTs and Services and statistic results are aggregated anonymously with other CERTs to show the average maturity reached.

Are reports available?

Sure. Reports are indispensable for Managements. At the end of each self-assessment, CERTrating provides for CERT and service the report with the overall evaluation and the suggested actions.

Is my rating history available?

In the "History" section of CERTrating it is possible to view all the Self-Assessment surveys completed during the use of CERTrating, thus having the possibility of keeping track of the progresses.

How soon should I reach a subsequent level of maturity?

ENISA suggests: "a growth path is suggested that reaches basic level within one year, intermediate two years later and advanced another two years later: a total of five years maximum.
Basic level already allows a minimum of successful co- operation between teams on incident handling, the higher levels are needed to allow the members of the CSIRT network to interact on all levels, including pro-actively, thus truly giving meaning to the word CSIRT Network."
ENISA CSIRT maturity assessment model, April 30, 2019, https://www.enisa.europa.eu/topics/csirts-in-europe/csirt-capabilities/csirt-maturity

What does "minimal actions" mean in CERTrating?

CERTrating suggests the minimum actions needed to move to the next level of maturity (e.g. from Not Basic to Basic). This approach is essential in order to progressively increase the overall maturity of its CERT, offering the possibility for users of identifying the area organization, resources, tools and processes in which are necessary improvements and associated investments.

What improvement/optimal actions are?

CERTrating suggests the best / optimal actions that should implement in your CERT in order to reach the highest level of maturity - "Advanced"-, the best maturity level provided by ENISA for CERTs.

Can I compare my CERT position with other CERTs?

One of the peculiarities of CERTrating is to show you how your CERT is ranked considering other CERTs in terms of CERT maturity level, services maturity level and the Performance Index.
These data are treated in a purely statistical manner.

Who can see my data?

Only accounts of your organization can directly check your data. The results will be made available to other companies in aggregated statistic data and anonymous form. This type of treatment for statistical purposes is essential to compare your CERT with the other CERTs, therefore to understand the maturity level of your organization with other CERTs guaranteeing anonymity.

How can I request the use of the platform?

You can use our dedicated "Contact" section and/or the Form created for all your needs. You could also send an email to Global Cyber Security Center (Viale Europa, 175 - 00144 Rome - Italy) at info@gcsec.org or by calling +39 06 59582258.