4 — Statistics and exposure¶
Theme: turning a marked record into numbers — task statistics, and the metrics used for occupational exposure assessment.
Everything here builds on the activities from 3 — Activities.
[1]:
import aerosoltools as at
elpi = at.load_elpi_file("../../tests/data/Sample_ELPI.txt")
elpi.mark_activities({
"Background": [("2023-09-07 09:06:50", "2023-09-07 09:07:50")],
"Emission": [("2023-09-07 09:07:55", "2023-09-07 09:08:30")],
"Decay": [("2023-09-07 09:09:00", "2023-09-07 09:10:50")],
})
elpi.activities
/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
/home/runner/work/aerosoltools/aerosoltools/src/aerosoltools/loaders/elpi.py:525: RuntimeWarning: ELPI density is not 1.0 g/cm3; bin edges were estimated from geometric means of CalculatedDi values.
return load_elpi_file_txt(
[1]:
['All data', 'Background', 'Emission', 'Decay']
What can this dataset be summarised on?¶
Which quantities are available depends on the instrument. A size-resolved instrument can produce mass and number fractions at any cut diameter; a single-channel one cannot. available_metrics reports what this particular dataset supports.
[2]:
for metric in elpi.available_metrics():
print(f"{metric.key:12s} {metric.label:28s} [{metric.unit}]")
PNC Number concentration [cm⁻³]
MASS Mass concentration [µg/m³]
PM1 PM1 [µg/m³]
PM2.5 PM2.5 [µg/m³]
PM4 PM4 [µg/m³]
PM10 PM10 [µg/m³]
One channel at a time¶
summarize gives descriptive statistics for a single channel, split by activity.
[3]:
elpi.summarize()
Summary of Total_conc:
+---+------------+---------+---------+---------+--------+--------------+
| | Segment | Min | Max | Mean | Std | N datapoints |
+---+------------+---------+---------+---------+--------+--------------+
| 0 | All data | 118.566 | 442.5 | 197.436 | 48.901 | 256 |
| 1 | Background | 137.253 | 442.5 | 222.08 | 61.048 | 62 |
| 2 | Emission | 138.416 | 314.828 | 202.937 | 42.281 | 37 |
| 3 | Decay | 121.176 | 309.717 | 179.173 | 39.193 | 112 |
+---+------------+---------+---------+---------+--------+--------------+
[3]:
| Segment | Min | Max | Mean | Std | N datapoints | |
|---|---|---|---|---|---|---|
| 0 | All data | 118.566 | 442.500 | 197.436 | 48.901 | 256 |
| 1 | Background | 137.253 | 442.500 | 222.080 | 61.048 | 62 |
| 2 | Emission | 138.416 | 314.828 | 202.937 | 42.281 | 37 |
| 3 | Decay | 121.176 | 309.717 | 179.173 | 39.193 | 112 |
Every activity at once¶
summarize_activities is the usual starting point: descriptive statistics per activity, including duration and — for size-resolved data — size metrics such as mode and geometric mean diameter.
[4]:
elpi.summarize_activities()
Summary of aerosol properties (transposed):
+------------------------+----------+------------+----------+--------+
| | All data | Background | Emission | Decay |
+------------------------+----------+------------+----------+--------+
| Duration (HH:MM) | 00:04 | 00:01 | 00:00 | 00:01 |
| PNC [cm⁻³] | 197.44 | 222.08 | 202.94 | 179.17 |
| PNC [cm⁻³] std | 48.9 | 61.05 | 42.28 | 39.19 |
| PM1 [µg/m³] | 1.29 | 1.4 | 1.31 | 1.2 |
| PM1 [µg/m³] std | 0.11 | 0.08 | 0.02 | 0.04 |
| PM2.5 [µg/m³] | 3.41 | 3.54 | 3.53 | 3.26 |
| PM2.5 [µg/m³] std | 0.18 | 0.15 | 0.1 | 0.1 |
| PM4 [µg/m³] | 5.77 | 5.63 | 6.22 | 5.7 |
| PM4 [µg/m³] std | 0.59 | 0.94 | 0.28 | 0.29 |
| PM10 [µg/m³] | 30.33 | 29.61 | 35.27 | 29.77 |
| PM10 [µg/m³] std | 9.32 | 15.83 | 2.71 | 3.96 |
| Total Mass [µg/m³] | 124.61 | 131.13 | 153.37 | 114.4 |
| Total Mass [µg/m³] std | 51.31 | 83.91 | 25.27 | 23.13 |
| Mode Dp [nm] | 125.2 | 116.4 | 129.8 | 134.3 |
| Mode Dp [nm] std | 131.3 | 127.5 | 131.6 | 136.2 |
| Median Dp [nm] | 224.2 | 211.8 | 227.8 | 228.6 |
| Median Dp [nm] std | 68.7 | 77.5 | 66.9 | 67.6 |
| GMD [nm] | 203.3 | 195.0 | 201.9 | 213.1 |
| GMD [nm] std | 79.7 | 80.7 | 77.9 | 84.2 |
+------------------------+----------+------------+----------+--------+
[4]:
| Segment | Duration (HH:MM) | PNC [cm⁻³] | PNC [cm⁻³] std | PM1 [µg/m³] | PM1 [µg/m³] std | PM2.5 [µg/m³] | PM2.5 [µg/m³] std | PM4 [µg/m³] | PM4 [µg/m³] std | PM10 [µg/m³] | PM10 [µg/m³] std | Total Mass [µg/m³] | Total Mass [µg/m³] std | Mode Dp [nm] | Mode Dp [nm] std | Median Dp [nm] | Median Dp [nm] std | GMD [nm] | GMD [nm] std | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | All data | 00:04 | 197.44 | 48.90 | 1.29 | 0.11 | 3.41 | 0.18 | 5.77 | 0.59 | 30.33 | 9.32 | 124.61 | 51.31 | 125.2 | 131.3 | 224.2 | 68.7 | 203.3 | 79.7 |
| 1 | Background | 00:01 | 222.08 | 61.05 | 1.40 | 0.08 | 3.54 | 0.15 | 5.63 | 0.94 | 29.61 | 15.83 | 131.13 | 83.91 | 116.4 | 127.5 | 211.8 | 77.5 | 195.0 | 80.7 |
| 2 | Emission | 00:00 | 202.94 | 42.28 | 1.31 | 0.02 | 3.53 | 0.10 | 6.22 | 0.28 | 35.27 | 2.71 | 153.37 | 25.27 | 129.8 | 131.6 | 227.8 | 66.9 | 201.9 | 77.9 |
| 3 | Decay | 00:01 | 179.17 | 39.19 | 1.20 | 0.04 | 3.26 | 0.10 | 5.70 | 0.29 | 29.77 | 3.96 | 114.40 | 23.13 | 134.3 | 136.2 | 228.6 | 67.6 | 213.1 | 84.2 |
You can restrict it to particular metrics or statistics rather than taking the default set.
[5]:
elpi.summarize_activities(metrics=["PNC", "PM2.5"], stats=["mean", "median", "max"])
Summary of aerosol properties (transposed):
+----------------------+----------+------------+----------+--------+
| | All data | Background | Emission | Decay |
+----------------------+----------+------------+----------+--------+
| Duration (HH:MM) | 00:04 | 00:01 | 00:00 | 00:01 |
| PNC [cm⁻³] | 197.44 | 222.08 | 202.94 | 179.17 |
| PNC [cm⁻³] median | 190.1 | 215.57 | 191.12 | 176.09 |
| PNC [cm⁻³] max | 442.5 | 442.5 | 314.83 | 309.72 |
| PM2.5 [µg/m³] | 3.41 | 3.54 | 3.53 | 3.26 |
| PM2.5 [µg/m³] median | 3.4 | 3.57 | 3.56 | 3.26 |
| PM2.5 [µg/m³] max | 3.95 | 3.95 | 3.73 | 3.6 |
+----------------------+----------+------------+----------+--------+
[5]:
| Segment | Duration (HH:MM) | PNC [cm⁻³] | PNC [cm⁻³] median | PNC [cm⁻³] max | PM2.5 [µg/m³] | PM2.5 [µg/m³] median | PM2.5 [µg/m³] max | |
|---|---|---|---|---|---|---|---|---|
| 0 | All data | 00:04 | 197.44 | 190.10 | 442.50 | 3.41 | 3.40 | 3.95 |
| 1 | Background | 00:01 | 222.08 | 215.57 | 442.50 | 3.54 | 3.57 | 3.95 |
| 2 | Emission | 00:00 | 202.94 | 191.12 | 314.83 | 3.53 | 3.56 | 3.73 |
| 3 | Decay | 00:01 | 179.17 | 176.09 | 309.72 | 3.26 | 3.26 | 3.60 |
Exposure assessment¶
summarize_exposure answers the occupational-hygiene question: given this measurement, what is the exposure, and how does it compare with a limit?
It reports a time-weighted average over twa_window (8 h by convention), the short-term exceedances over short_window (15 min by convention), peak counts, high percentiles, and the time spent above each limit.
[6]:
exposure = elpi.summarize_exposure(
metric="PM4.2", # respirable dust
activities=["Emission"],
background="Background", # subtract the mean of this activity
exposure_hours=None, # None -> use the measured task duration
short_limit=1.0, # STEL, in metric units
long_limit=1.0, # 8 h OEL, in metric units
short_window="15min",
twa_window="8h",
)
exposure
Exposure summary by segment for metric 'PM4.2' (µg/m³):
+---------------------------------+----------+
| Metric | Emission |
+---------------------------------+----------+
| Metric | PM4.2 |
| Duration [HH:MM] | 00:00 |
| Max [µg/m³] | 7.473 |
| 95th percentile [µg/m³] | 7.215 |
| 75th percentile [µg/m³] | 6.951 |
| 50th percentile [µg/m³] | 6.648 |
| 25th percentile [µg/m³] | 6.557 |
| 5th percentile [µg/m³] | 6.276 |
| Peaks [count] | 0 |
| STEL [µg/m³] | 1.0 |
| STEL window [offset] | 15min |
| STEL exceedance [min] | 0.02 |
| STEL episodes [count] | 1 |
| Exposure limit [µg/m³] | 1.0 |
| Exposure limit exceedance [min] | 0.62 |
| TWA concentration [µg/m³] | 6.018 |
| TWA window [offset] | 8h |
+---------------------------------+----------+
[6]:
| Segment | Metric | Duration [HH:MM] | Max [µg/m³] | 95th percentile [µg/m³] | 75th percentile [µg/m³] | 50th percentile [µg/m³] | 25th percentile [µg/m³] | 5th percentile [µg/m³] | Peaks [count] | STEL [µg/m³] | STEL window [offset] | STEL exceedance [min] | STEL episodes [count] | Exposure limit [µg/m³] | Exposure limit exceedance [min] | TWA concentration [µg/m³] | TWA window [offset] | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Emission | PM4.2 | 00:00 | 7.473 | 7.215 | 6.951 | 6.648 | 6.557 | 6.276 | 0 | 1.0 | 15min | 0.02 | 1 | 1.0 | 0.62 | 6.018 | 8h |
Three arguments carry most of the meaning:
``background`` — either a number, or the name of an activity whose average is used. Subtracting a measured background separates the process contribution from what was already in the room.
``exposure_hours`` — how long the worker is assumed to be exposed.
Noneuses the measured duration of the activity; give a number to extrapolate a short measurement to a full shift.``metric`` — which quantity the limit applies to.
Compare a short task assumed to last the whole shift against the same task taken at its measured length:
[7]:
measured = elpi.summarize_exposure(metric="PM4.2", activities=["Emission"],
background="Background", exposure_hours=None)
full_shift = elpi.summarize_exposure(metric="PM4.2", activities=["Emission"],
background="Background", exposure_hours=8.0)
twa_cols = [c for c in measured.columns if "TWA" in c]
print("assumed duration = measured task length")
print(measured[["Segment"] + twa_cols].to_string(index=False))
print()
print("assumed duration = 8 h")
print(full_shift[["Segment"] + twa_cols].to_string(index=False))
Exposure summary by segment for metric 'PM4.2' (µg/m³):
+---------------------------------+----------+
| Metric | Emission |
+---------------------------------+----------+
| Metric | PM4.2 |
| Duration [HH:MM] | 00:00 |
| Max [µg/m³] | 7.473 |
| 95th percentile [µg/m³] | 7.215 |
| 75th percentile [µg/m³] | 6.951 |
| 50th percentile [µg/m³] | 6.648 |
| 25th percentile [µg/m³] | 6.557 |
| 5th percentile [µg/m³] | 6.276 |
| Peaks [count] | 0 |
| STEL [µg/m³] | 1.0 |
| STEL window [offset] | 15min |
| STEL exceedance [min] | 0.02 |
| STEL episodes [count] | 1 |
| Exposure limit [µg/m³] | 1.0 |
| Exposure limit exceedance [min] | 0.62 |
| TWA concentration [µg/m³] | 6.018 |
| TWA window [offset] | 8h |
+---------------------------------+----------+
Exposure summary by segment for metric 'PM4.2' (µg/m³):
+---------------------------------+----------+
| Metric | Emission |
+---------------------------------+----------+
| Metric | PM4.2 |
| Duration [HH:MM] | 00:00 |
| Max [µg/m³] | 7.473 |
| 95th percentile [µg/m³] | 7.215 |
| 75th percentile [µg/m³] | 6.951 |
| 50th percentile [µg/m³] | 6.648 |
| 25th percentile [µg/m³] | 6.557 |
| 5th percentile [µg/m³] | 6.276 |
| Peaks [count] | 0 |
| STEL [µg/m³] | 1.0 |
| STEL window [offset] | 15min |
| STEL exceedance [min] | 0.02 |
| STEL episodes [count] | 1 |
| Exposure limit [µg/m³] | 1.0 |
| Exposure limit exceedance [min] | 0.62 |
| TWA concentration [µg/m³] | 6.722 |
| TWA window [offset] | 8h |
+---------------------------------+----------+
assumed duration = measured task length
Segment TWA concentration [µg/m³] TWA window [offset]
Emission 6.018 8h
assumed duration = 8 h
Segment TWA concentration [µg/m³] TWA window [offset]
Emission 6.722 8h
Choosing a metric¶
For size-resolved data the metric can be any of:
"PNC"— total number concentration"MASS"— total mass concentration"PM<x>","PN<x>","PS<x>","PV<x>"— cumulative mass, number, surface or volume below cut diameter<x>in µm, e.g."PM2.5""PM<a>-<b>"— the band between two diameters, e.g."PM1-4", computed with the EN 481 / ISO 7708 penetration curves
Several activities can be summarised in one call, which is the convenient form when comparing tasks.
[8]:
elpi.summarize_exposure(
metric="PM10",
activities=["Emission", "Decay"],
background="Background",
long_limit=5.0,
short_limit=10.0,
)
Exposure summary by segment for metric 'PM10' (µg/m³):
+---------------------------------+----------+-------+
| Metric | Emission | Decay |
+---------------------------------+----------+-------+
| Metric | PM10 | PM10 |
| Duration [HH:MM] | 00:00 | 00:01 |
| Max [µg/m³] | 0.246 | 0.208 |
| 95th percentile [µg/m³] | 0.194 | 0.153 |
| 75th percentile [µg/m³] | 0.165 | 0.126 |
| 50th percentile [µg/m³] | 0.154 | 0.111 |
| 25th percentile [µg/m³] | 0.139 | 0.099 |
| 5th percentile [µg/m³] | 0.121 | 0.084 |
| Peaks [count] | 1 | 0 |
| STEL [µg/m³] | 10.0 | 10.0 |
| STEL window [offset] | 15min | 15min |
| STEL exceedance [min] | 0.0 | 0.0 |
| STEL episodes [count] | 0 | 0 |
| Exposure limit [µg/m³] | 5.0 | 5.0 |
| Exposure limit exceedance [min] | 0.0 | 0.0 |
| TWA concentration [µg/m³] | 0.132 | 0.131 |
| TWA window [offset] | 8h | 8h |
+---------------------------------+----------+-------+
[8]:
| Segment | Metric | Duration [HH:MM] | Max [µg/m³] | 95th percentile [µg/m³] | 75th percentile [µg/m³] | 50th percentile [µg/m³] | 25th percentile [µg/m³] | 5th percentile [µg/m³] | Peaks [count] | STEL [µg/m³] | STEL window [offset] | STEL exceedance [min] | STEL episodes [count] | Exposure limit [µg/m³] | Exposure limit exceedance [min] | TWA concentration [µg/m³] | TWA window [offset] | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Emission | PM10 | 00:00 | 0.246 | 0.194 | 0.165 | 0.154 | 0.139 | 0.121 | 1 | 10.0 | 15min | 0.0 | 0 | 5.0 | 0.0 | 0.132 | 8h |
| 1 | Decay | PM10 | 00:01 | 0.208 | 0.153 | 0.126 | 0.111 | 0.099 | 0.084 | 0 | 10.0 | 15min | 0.0 | 0 | 5.0 | 0.0 | 0.131 | 8h |
Single-channel instruments¶
A 1D instrument has no size distribution, so the metric is its own measurement — "PNC" for a CPC.
[9]:
cpc = at.load_cpc_file("../../tests/data/Sample_CPC_AIM.txt")
cpc.mark_activities({
"Task": [("2023-08-14 11:14:00", "2023-08-14 11:18:00")],
})
cpc.summarize_exposure(
metric="PNC",
activities=["Task"],
exposure_hours=8.0,
long_limit=1e4,
short_limit=2e4,
)
Exposure summary by segment for metric 'PNC' (cm⁻³) [1D]:
+---------------------------------+----------+
| Metric | Task |
+---------------------------------+----------+
| Metric | PNC |
| Duration [HH:MM] | 00:03 |
| Max [cm⁻³] | 1559.0 |
| 95th percentile [cm⁻³] | 1511.0 |
| 75th percentile [cm⁻³] | 1457.0 |
| 50th percentile [cm⁻³] | 1415.0 |
| 25th percentile [cm⁻³] | 1366.5 |
| 5th percentile [cm⁻³] | 1307.0 |
| Peaks [count] | 0 |
| STEL [cm⁻³] | 20000.0 |
| STEL window [offset] | 15min |
| STEL exceedance [min] | 0.0 |
| STEL episodes [count] | 0 |
| Exposure limit [cm⁻³] | 10000.0 |
| Exposure limit exceedance [min] | 0.0 |
| TWA concentration [cm⁻³] | 1412.165 |
| TWA window [offset] | 8h |
+---------------------------------+----------+
[9]:
| Segment | Metric | Duration [HH:MM] | Max [cm⁻³] | 95th percentile [cm⁻³] | 75th percentile [cm⁻³] | 50th percentile [cm⁻³] | 25th percentile [cm⁻³] | 5th percentile [cm⁻³] | Peaks [count] | STEL [cm⁻³] | STEL window [offset] | STEL exceedance [min] | STEL episodes [count] | Exposure limit [cm⁻³] | Exposure limit exceedance [min] | TWA concentration [cm⁻³] | TWA window [offset] | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Task | PNC | 00:03 | 1559.0 | 1511.0 | 1457.0 | 1415.0 | 1366.5 | 1307.0 | 0 | 20000.0 | 15min | 0.0 | 0 | 10000.0 | 0.0 | 1412.165 | 8h |
Saving the result¶
Both summary functions accept filename and append to a CSV or Excel file, so results from several measurements accumulate in one place.
elpi.summarize_activities(filename="campaign_summary.xlsx",
sheet_name="2023-09-07")
Next: 5 — Plotting.