Data Engineering & Infrastructure

PySpark Fundamentals: Distributed Data Processing at Scale

Master PySpark DataFrame API, Spark SQL, and distributed processing patterns for building production

Partitioning Strategies: Optimizing Data Layout for Performance

Learn how partitioning strategies affect query performance, storage efficiency, and cost in modern d

Building Resilient Data Pipelines with Spark and dbt

Combine Spark's processing power with dbt's transformation layer to build production-grade, tested,

Data Contracts & Data Lineage: Building Trust in Modern Data Pipelines

Data contracts formalize the agreement between data producers and consumers, while data lineage trac

Data Mesh: Architecture, Governance, and Real-World Adoption

Data mesh applies domain-driven design to data architecture, organizing ownership by business domain

Change Data Capture: Real-Time Sync Patterns and Tools

Change Data Capture (CDC) captures row-level changes in databases and streams them to downstream sys

All Articles

Monitoring Mitochondrial Function With High-Resolution Respirometry During Ex-Situ Heart Perfusion.

Due to the high adenosine-triphosphate (ATP) consumption of the myocardial tissue, cardiac function is dependent on efficient mitochondrial respiration. Ischemia and reperfusion injury (I/R) has a severe impact on the function of the mitochondrial respiratory system. Ex-situ heart perfusion (ESHP) c...

Nov 1, 2026

Crosstalk between mitophagy and metabolism in cancer: Mechanisms and therapeutic opportunities.

Mitophagy is a highly selective autophagic process that eliminates dysfunctional mitochondria to enforce stringent cellular quality control, acting as a context-dependent modulator of stress adaptation in cancer cells. Concurrently, to thrive in hostile microenvironments, cancer cells undergo extens...

Nov 1, 2026

Finding Competitive Equilibria in Online Marketplaces with Platform Fees, Taxes, and Subsidies

We study equilibrium computation in multi-unit trading networks with valuations satisfying generalized full substitutes and agent- and transaction-specific piecewise-linear transfer functions. These t

Tessera: Fast Join-Aggregates over Skewed Sparse Relations in Database Engines

Join-aggregate queries count or sum matching rows for each pair of values, often over skewed sparse relations in which a few values are frequent and most pairs of values never co-occur. Conventional e

On Zeros and Algorithms for Disordered Systems II: The Antiferromagnetic Ising Model up to Kesten-Stigum

We give the first polynomial-time algorithm to estimate the partition function of the antiferromagnetic Ising model on the uniformly random $d$-regular graph to arbitrarily high accuracy for all inver

When Imputation Changes the Nearest Neighbor: Exact Certification of Similarity Retrieval for Incomplete Sensor Time Series

Sensor time series routinely reach similarity search with missing values, and the retrieved neighbours then depend on how the gaps were filled. We ask which retrieval conclusions hold for every admiss

Nearly-Optimal Time-Sample Trade-off in Consensus Dynamics

How does the number of sampled opinions affect the time required to reach consensus in distributed consensus dynamics? We study the synchronous $h$-\textsf{Majority}, in which every node independently

Power Adjusting Games Under Block-Dependent Pool Payout Schemes

Advanced power adjusting block withholding attacks such as PAW and T-PAW have been analyzed so far under a global-reset version of the proportional scheme (GRP), which is rarely used in practice. In t

Exact Competitive Ratios of Index Rules for Scheduling with Obligatory Testing

We revisit scheduling with obligatory testing, where each job consists of two operations: a test with an initially known time and an operation whose processing time is revealed upon completion of the

Zero-freeness for Random Regular Graphs up to the Replica Symmetry Breaking Threshold

We study antiferromagnetic spin systems on random regular graphs through the lens of the phase transitions characterized by the complex zeros of the partition function. Our motivating examples include

RecHash: Segmented Hash-Entangled Chains for Stateful Sequential Authentication

RecHash is a stateful sequential authentication construction based on paired hash-chain openings and an authenticated record history. It develops the decreasing position relation of the 2019 hash-enta

AI in the EGI e-infrastructure -- Past experiences and future strategies

The EGI Federation has provided federated e-infrastructure services to European and global science for over 15 years. We review EGI's early experiences with AI in the interTwin and iMagine Horizon Eur

SPECTRUM: A Strategic Framework and Technical Blueprint for European Exascale Research Data and Compute Infrastructure

The HiLumi LHC, the SKA Observatory and LOFAR 2.0 will take High Energy Physics (HEP) and Radio Astronomy (RA) towards exabyte-scale data within the next decade, beyond the capacity of today's infrast

Can We Rely on Language Models to Orchestrate Services across the Edge-Cloud Continuum?

Service orchestration across the edge--cloud continuum increasingly uses language models (LMs) to translate intents and operational evidence into scheduling, configuration, diagnosis, and remediation

Reinforcement learning for control of partial differential equations

Controlling dynamical systems governed by partial differential equations (PDEs) remains a challenging task in scientific computing, owing to high-dimensional state spaces, distributed actuation, nonli

NECA: Neighbor Event Curve Augmentation for Multi-Horizon ICU Deterioration Prediction

Early prediction of ICU organ dysfunction events supports timely clinical monitoring and intervention, but estimating short-term organ-specific risk from early ICU records remains challenging due to i

Deep Learning Based Behavioral Drift Modeling for Continuous Biometric Authentication

This research proposes a two-level keystroke dynamics authentication system that provides defense-in-depth security through continuous behavioral drift monitoring. Level 1 employs fixed-text password

Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models

Language models often need to maintain multiple contextual variables that evolve independently within a document. Prior work has identified structured representations of individual temporal or context

How to Loop Your Transformer: Loop Scaling, Specialization, and Parallel Inference

Looping in transformer models reuses weights by repeatedly applying the same transformer, or parts of it, making the model deeper without increasing its parameter count. Whether looping is useful depe

True 3D Holography: A Communication Service of Tomorrow and Its Requirements for a New Converged Cloud and Network Architecture on the Path to 6G

Research project 6G NeXt is considering true 3D holography as a use case, setting requirements on the communication as well as the computing infrastructure. In a future holographic communication servi

Only Accepted Tokens Count: An Energy Accounting of Speculative Decoding Under Latency Targets

Large language model serving is limited by facility power. Energy per token determines how many tokens a datacenter can serve, while a latency target bounds how quickly each user receives them. During

Aperiodic EEG features of cognition: a systematic review.

A defining feature of electroencephalography (EEG) data is its 1/ f -like spectral structure, whereby power decreases as frequency increases. The nonoscillatory, aperiodic activity underlying this structure has regained attention as an index of cortical excitation-inhibition balance or neural noise,...

Oct 26, 2026

Beyond Banks versus Markets: Financial Structure and Sustainable Debt in Europe

We examine how the financial structure underlying the EU’s Savings and Investments Union is associated with sustainable corporate debt issuance. Using transaction-level data on bonds and syndicated loans over 2019–2023 and Poisson pseudo-maximum likelihood estimation, we distinguish use-of-proceeds...

Oct 25, 2026