Research Methodology
Key Insights
- How AcaciaFund transforms raw signals into quality-scored research โ SQI, Bloom taxonomy, pipeline stages.
How AcaciaFund transforms raw signals into structured, quality-scored research articles.
Pipeline Stages
1. Source Ingestion
Content is drawn from three primary feeds:
- HackerNews API (news.ycombinator.com) โ technology, business, and science stories ranked by community engagement (points, comments). Top ~30 stories per day are analyzed.
- arXiv API (arxiv.org) โ academic preprints across computer science, physics, mathematics, quantitative biology, and finance. Filtered by relevance to pillar topics.
- PubMed โ biomedical literature (archived โ science pillar discontinued).
2. Entity Extraction & Summarization
Each source is processed through NLP pipelines for:
- Named Entity Recognition (NER) โ extracting organizations, people, technologies, regulations, and financial instruments.
- Automatic Summarization โ extractive summarization identifying key claims, numerical data, and conclusions.
- Topic Modeling โ LDA-based classification into pillar-relevant topics and subtopics.
3. Bloom Taxonomy Classification
Every article is assessed across six cognitive levels:
| Level | Description | Example Question |
|---|---|---|
| Remember | Recall facts and basic concepts | Which pillar does this article belong to? |
| Understand | Explain ideas and concepts | What is the primary domain of this article? |
| Apply | Use information in new situations | How can these findings be applied in practice? |
| Analyze | Draw connections among ideas | What assumptions underlie this analysis? |
| Evaluate | Justify a stand or decision | How strong is the evidence presented? |
| Create | Produce new or original work | Design an experiment to test this hypothesis. |
4. Signal Quality Index (SQI) Computation
SQI is a composable metric computed per article as a weighted combination:
SQI = w1 ยท Authority + w2 ยท Freshness + w3 ยท Consensus + w4 ยท Relevance where: Authority = domain reputation score (0โ1) Freshness = 1 - (days_since_publication / freshness_window) Consensus = cross-source agreement rate Relevance = keyword overlap with pillar taxonomy Default weights: w1=0.35, w2=0.25, w3=0.25, w4=0.15
Weights are adjustable per pillar to prioritize different quality dimensions (e.g., AML favors authority, Data Engineering favors freshness).
5. Cross-Pillar Analysis
Each article is analyzed for connections to other pillars via:
- Shared entity references โ organizations, regulations, or technologies that appear across pillars
- Source overlap โ articles from different pillars citing the same source
- Topic bridging โ latent topic modeling revealing cross-domain themes
6. Quality Gates & Serving
Before publication, each article passes through quality gates:
- Schema validation โ Pydantic ensures all required fields are present and correctly typed
- Source diversity check โ minimum source count and domain diversity thresholds
- SQI threshold โ minimum SQI of 0.35 for publication
- Deterministic build โ same registry.json always produces identical output
The final artifact is static HTML deployed to Cloudflare Pages via python3.13 build.py.
Last updated: 2026-06-08
Article Metadata
Further Reading
Databricks Blog
Lakehouse, Spark, Delta Lake, Unity Catalog โ engineering blog
Apache Kafka
Kafka documentation, KIPs, and ecosystem updates
Apache Flink
Flink documentation and release notes
Apache Iceberg
Iceberg table format โ specs, REST catalog, performance
dbt Blog
dbt Labs engineering blog โ analytics engineering, Semantic Layer
Dagster Blog
Dagster orchestration โ software-defined assets, IO managers
Feynman Concept Cards
Master each concept: read the ELI5, explore analogies, work examples, and teach it back.
DataOps is a best practices concept in the Data Engineering pillar.
Analogy
Example
Find Gaps
Explain DataOps as if teaching a colleague who is new to best practices. Cover: what it is, how it works, and why it matters.
Create
Create a checklist that demonstrates DataOps in a real-world best practices scenario. Walk through your design decisions.
Show solution
A checklist for DataOps should include: 1. The core components of dataops 2. How they interact 3. Expected outcomes or outputs
Research is a specialized concept in the Markets pillar.
Analogy
Example
Find Gaps
Explain Research as if teaching a colleague who is new to specialized. Cover: what it is, how it works, and why it matters.
Create
Create a diagram that demonstrates Research in a real-world specialized scenario. Walk through your design decisions.
Show solution
A diagram for Research should include: 1. The core components of research 2. How they interact 3. Expected outcomes or outputs