WP P - Monte Carlo Productions¶
The production of simulated sample in LHCb is the major user of CPU in the LHCb experiment, and given the large variety of samples necessitated by the physics program of the experiment, it requires its own work package for maintenance, operation and improvement. Monte Carlo productions are one major point of contact between the Simulation Project and the rest of the experiment. For this reason, close contact with the LHCb Computing, RTA and DPA projects is essential to ensure the necessary infrastructure and processing functionality is provided. Tight connection with PWGs is also essential to guarantee their requirements are reasonably met.
Mandate¶
The mandate of this work package, WP P is:
To ensure maintenance and improvement of Production Models for LHCb Simulation.
To provide first point of contact for management of the Simulation. Productions of the LHCb experiment and intervene in case of problems.
To maintain communication with the LHCb Computing Project and LHCbDIRAC team.
To ensure open communication channels with RTA and DPA projects for simulation productions.
To maintain and improve the simulation request processes, including filtered productions.
To manage and publish generator statistics.
To coordinate with WP D, WP F and WP G to ensure production capabilities for current and future frameworks.
To support Run 1 & 2 simulations until needed
The WP is responsible for the management and release of the MCStatTools package, with scripts to extract and publish the generator statistical quantities from the production and provides tools and websites with easy access to the information by analysts.
Developments¶
LbMCSubmit¶
LbMCSubmit is a python package for defining, testing and submitting simulation production requests from a YAML file. This is intended to be a replacement for the “request models” that reside in LHCbDIRAC, which have become increasingly time-consuming to maintain and update as their number has grown, and are prone to typos and copy/paste mistakes in the process.
Rather than a system of templates, LbMCSubmit creates requests and steps from
procedures defines in python functions, which helps ensure consistency and
reduces the workload involved in their maintenance.
Where so desired, it has the ability to always use the latest version of LHCb
applications or data-packages matching a certain patter. For example: the latest
Gen/DecFiles.v30r* version is taken when creating Sim09 requests.
For instructions on how to use LbMCSubmit to create production requests, see Creating requests with LbMCSubmit.
SimDQ¶
Checking the quality of simulation samples early on in their production is crucial for avoiding wasted computing time, particularly as the sample sizes grow with increased luminosity. The Simulation Data Quality (SimDQ) system displays histograms produced in test jobs (run at the start of a production) to a shifter via the Monet application, which is also used for Online and Offline Data Quality. The shifter can flag problems and signal for buggy productions to be halted early.
- SimDQ consists of three parts:
the Histogram Merging Agent running in LHCbDIRAC, which periodically checks for completed test productions and writes metadata about the histogram files;
the SimDQ data repository, which hosts the metadata and deduces appropriate samples to use as reference for each new production;
the SimDQ tab in Monet, which reads the metadata from the repository and fetches & renders the histograms.
The infrastructure is planned to be adapted for Offline Data Quality, where the histograms are produced in Sprucing test jobs.