# Privacy Stack (`bikram786/privacy-stack`) Actor Privacy researcher & developer building production Apify actors for arXiv privacy research. Privacy Stack brings 1 5,00+ real arXiv privacy papers into one place ..carefully verified with no fake URLs & no duplicates. Categories : Internet Privacy Data Privacy Crypto Privacy Post-Quantum Privacy - **URL**: https://apify.com/bikram786/privacy-stack.md - **Developed by:** [Bikram Biswas](https://apify.com/bikram786) (community) - **Categories:** Developer tools, AI, Integrations - **Stats:** 9 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.10 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Privacy Stack 🔐 – 5000 Real arXiv Privacy Papers for Researchers & Builders **Ultimate Privacy Research Scraper – Converts peer‑reviewed privacy & anonymity papers into clean, reproducible, auditable datasets ready for analysis, dashboards, and code.** Live Actor: https://console.apify.com/oblate_wildcat/privacy-stack GitHub: https://github.com/BikramBiswas786/privacy-stack --- ### 💡 What Is Privacy Stack? Privacy Stack is an Apify Actor that builds a **large‑scale, high‑quality research corpus** of real arXiv papers in security & privacy. It scrapes and normalizes **5000 unique arXiv cs.CR papers** across **4 critical categories**, making it easy to explore, filter, and build on top of the latest privacy research without touching the arXiv UI. [web:20][file:13] You get: - A **clean JSON/CSV dataset** you can drop into analysis pipelines - Strong **deduplication guarantees** - A **stable schema** designed for LLMs, dashboards, and downstream tools --- ### 📊 Categories (4 × 1250 Papers) Each run targets exactly **1250 papers per category**, for a total of **5000 unique cs.CR papers**: 1. **🌐 Internet Privacy** - Tor, mix networks, I2P, VPNs, onion routing - Traffic analysis attacks & defenses - Website fingerprinting, metadata‑hiding systems 2. **🔐 Crypto Privacy** - Zero‑knowledge proofs (zk‑SNARKs, zk‑STARKs) - FHE, MPC, Bulletproofs, Pedersen commitments - Privacy coins (Zcash, Monero), mixer protocols, CoinJoin 3. **📊 Data Privacy** - Differential privacy (local & global) - Federated learning, secure aggregation - Synthetic data, re‑identification resistance, anonymization 4. **⚛️ Post‑Quantum / PQ Security** - Kyber, Dilithium, SPHINCS+, Falcon - Lattice‑based crypto, hash‑based signatures - PQ‑safe anonymous communication & key exchange Each paper is tagged with a **primary category** plus the **full arXiv category string**, so you can slice the dataset however you want. [file:13] --- ### 🚀 Key Features - ✅ **100% real arXiv papers** Directly scraped from arxiv.org `cs.CR` – no synthetic titles, no hallucinations, no fake IDs. - ✅ **5000 UNIQUE papers** Global deduplication by arXiv ID, plus **per‑category deduplication** so the same paper is never counted twice within a category. - ✅ **Balanced categories** 1250 papers for each of the 4 categories → balanced training/test sets for ML and fair comparisons between research areas. - ✅ **Production‑grade dataset schema** Designed for: - LLM context building - dashboards (Grafana/Metabase/Superset) - offline analytics (Python/pandas, DuckDB, BigQuery) - ✅ **Zero manual setup on Apify** No `requirements.txt` needed – runs on Apify’s managed Python runtime. - ✅ **Repeatable & auditable** Same input → same structure, easy to diff across runs as new papers appear on arXiv. --- ### 🧱 Dataset Schema Each paper in the dataset has a consistent JSON structure: ```json { "id": 1, "title": "Device-Independent Anonymous Communication", "arxiv_id": "2512.21047", "full_category": "cs.CR (Internet Privacy)", "short_category": "internet_privacy", "authors": ["John Doe", "Jane Smith"], "url": "https://arxiv.org/abs/2512.21047", "pdf_url": "https://arxiv.org/pdf/2512.21047.pdf", "is_real_arxiv": true, "published": "2025-12-21", "updated": "2025-12-23", "abstract": "We propose a device-independent protocol for anonymous communication...", "source_run_id": "RUN_ID_FOR_AUDIT" } ```` *** ### 📊 Apify Console Output Tabs When you run Privacy Stack in Apify Console, the **Output** tab is split into multiple views (using dataset schema): \[file:15] - **📚 All Papers (5000)** – full corpus merged - **🌐 Internet Privacy (1250)** – Tor, mixnets, I2P, traffic analysis - **🔐 Crypto Privacy (1250)** – ZK, FHE, MPC, crypto protocols - **📊 Data Privacy (1250)** – DP, FL, anonymization, re‑identification - **⚛️ Post‑Quantum (1250)** – Kyber, Dilithium, PQ anonymous systems - **📋 Live Logs** – scrape progress, dedup stats, category counts Each view is sortable & filterable directly in the Apify Console, and also accessible as CSV/JSON via API. *** ### 📥 Sample Output Snippet ```json { "id": 42, "title": "Traffic Analysis Resistant Mix Networks for the Modern Internet", "arxiv_id": "2507.12345", "full_category": "cs.CR (Cryptography and Security)", "short_category": "internet_privacy", "authors": ["Alice Anon", "Bob Mixnet"], "url": "https://arxiv.org/abs/2507.12345", "pdf_url": "https://arxiv.org/pdf/2507.12345.pdf", "is_real_arxiv": true } ``` *** ### ⚙️ How It Works (High‑Level) 1. **Input**: categories + maximum papers per category (defaults to 1250 × 4). 2. **Fetch arXiv feeds / search results** for each category (`cs.CR` + keywords / sub-tags). 3. **Normalize results** into the unified schema: - parse title, authors, IDs, URLs, dates, category strings 4. **Deduplicate**: - global deduplication by `arxiv_id` - ensure each category’s slice has only unique entries 5. **Store** into Apify Dataset with **multiple views** (all + per‑category). The Actor is designed to be **idempotent** in terms of structure, but you will naturally see **newer papers** when you re‑run it over time. *** ### 🚀 Quick Start #### 1. Run from Apify Console 1. Open: **Privacy Stack Actor**\ `https://console.apify.com/oblate_wildcat/privacy-stack` 2. Set input (optional): - `maxPapersPerCategory`: default 1250 - category toggles (if you want only 1–2 categories) 3. Click **Run** 4. When it finishes, open the **Output** tab: - Browse `All Papers` - Or switch to specific category views 5. Export as: - **JSON** (`items?clean=true`) - **CSV** (`items?format=csv`) - **HTML table** (for quick browsing) #### 2. Run via Apify CLI ```bash apify run privacy-stack-research-scraper ``` This will: - run the Actor locally - store dataset in `./storage/datasets/default/` - you can then inspect `OUTPUT.json` or CSV in that folder. *** ### 🧪 Example: Using the Dataset in Python ```python import requests import pandas as pd DATASET_URL = "https://api.apify.com/v2/datasets//items?clean=true" res = requests.get(DATASET_URL) res.raise_for_status() items = res.json() df = pd.DataFrame(items) ## Example: show recent ZK papers zk_df = df[df['title'].str.contains("zero-knowledge", case=False, na=False)] print(zk_df[['title', 'arxiv_id', 'url']].head()) ## Example: count papers per short_category print(df['short_category'].value_counts()) ``` *** ### 🧠 Typical Use Cases - **Literature review** for PhD / MSc / paper writing\ Quickly get 5000+ relevant cs.CR papers organized by topical area. - **Benchmark building**\ Curate evaluation sets for LLMs, anonymization tools, or privacy frameworks. - **Trend analysis**\ See how research volume changes over time in areas like ZK proofs or post‑quantum crypto. - **Dataset for downstream models**\ Use `title + abstract` as input for topic modeling, embeddings, or semantic search. - **Meta‑research**\ Study the evolution of anonymity, privacy‑preserving ML, and PQ crypto. *** ### 🔐 Design Principles - **Real papers only** – every record must correspond to a real arXiv entry. - **Transparent scraping** – URLs always point back to arxiv.org. - **No guessing / hallucinating metadata** – if arXiv does not provide it, it is not faked. - **Reproducibility** – input + time window → deterministically shaped dataset schema. *** ### 📦 Actor Input (Suggested Schema) Typical input fields (simplified): ```json { "maxPapersPerCategory": 1250, "includeInternetPrivacy": true, "includeCryptoPrivacy": true, "includeDataPrivacy": true, "includePostQuantum": true } ``` You can extend this in future (e.g., year range, specific arXiv query strings, exclusion filters). *** ### 🧑‍💻 About the Author **Bikram Biswas** (`@BikramBiswas786`) - Quantum & privacy tooling developer - Creator of **Anon Lab** (interactive privacy paper explorer) - Active on Apify building research‑grade Actors for security, privacy, and data aggregation. Apify profile: https://apify.com/bikrambiswas *** ### 📄 Citation If Privacy Stack helps in your work, you can cite it as: ```bibtex @software{biswas2025privacystack, author = {Biswas, Bikram}, title = {Privacy Stack: 5000 Real arXiv Privacy Papers for Researchers}, year = {2025}, url = {https://apify.com/oblate_wildcat/privacy-stack} } ``` *** ### 📝 License & Ethics - Use this dataset **responsibly**. - All papers belong to their respective authors and arXiv. - This Actor only organizes metadata and links; it does not strip or redistribute paywalled content. *** **Privacy Stack turns scattered security & privacy literature into a single, structured research surface you can actually build on.**\ Run it, export it, and plug it straight into your research pipeline. # Actor input Schema ## `sources` (type: `array`): List of sources and starting URLs to scrape ## `maxPapersPerSource` (type: `integer`): Maximum number of papers to extract from each source ## `includeImplementations` (type: `boolean`): Whether to extract and include Python implementations if available ## `extractAbstracts` (type: `boolean`): Extract paper abstracts and key concepts ## `timeout` (type: `integer`): Timeout for HTTP requests in seconds ## Actor input object example ```json { "sources": [ { "name": "arXiv Privacy", "url": "https://arxiv.org/list/cs.CR/recent", "category": "cryptography" }, { "name": "ACM CCS", "url": "https://www.sigsac.org/ccs/", "category": "security" }, { "name": "IEEE S&P", "url": "https://www.ieee-security.org/", "category": "privacy" } ], "maxPapersPerSource": 50, "includeImplementations": true, "extractAbstracts": true, "timeout": 30 } ``` # Actor output Schema ## `allPapers` (type: `string`): Complete dataset of 5000 unique privacy research papers ## `internetPrivacy` (type: `string`): Tor, mixnets, anonymous routing papers ## `cryptoPrivacy` (type: `string`): ZK, FHE, MPC research papers ## `dataPrivacy` (type: `string`): Differential Privacy, Federated Learning papers ## `postQuantum` (type: `string`): Kyber, Dilithium, lattice crypto papers ## `runLog` (type: `string`): Scraping progress and statistics # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("bikram786/privacy-stack").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("bikram786/privacy-stack").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call bikram786/privacy-stack --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=bikram786/privacy-stack", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Privacy Stack", "description": "Privacy researcher & developer building production Apify actors for arXiv privacy research.\n\nPrivacy Stack brings 1 5,00+ real arXiv privacy papers into one place ..carefully verified with no fake URLs & no duplicates.\n\nCategories : \nInternet Privacy\nData Privacy\nCrypto Privacy\nPost-Quantum Privacy", "version": "0.0", "x-build-id": "uD7uahxmSjv44JJML" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/bikram786~privacy-stack/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-bikram786-privacy-stack", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/bikram786~privacy-stack/runs": { "post": { "operationId": "runs-sync-bikram786-privacy-stack", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/bikram786~privacy-stack/run-sync": { "post": { "operationId": "run-sync-bikram786-privacy-stack", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "sources" ], "properties": { "sources": { "title": "Research Paper Sources", "type": "array", "description": "List of sources and starting URLs to scrape", "default": [ { "name": "arXiv Privacy", "url": "https://arxiv.org/list/cs.CR/recent", "category": "cryptography" }, { "name": "ACM CCS", "url": "https://www.sigsac.org/ccs/", "category": "security" }, { "name": "IEEE S&P", "url": "https://www.ieee-security.org/", "category": "privacy" } ] }, "maxPapersPerSource": { "title": "Max Papers Per Source", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of papers to extract from each source", "default": 50 }, "includeImplementations": { "title": "Include Code Implementations", "type": "boolean", "description": "Whether to extract and include Python implementations if available", "default": true }, "extractAbstracts": { "title": "Extract Abstracts", "type": "boolean", "description": "Extract paper abstracts and key concepts", "default": true }, "timeout": { "title": "Request Timeout (seconds)", "minimum": 5, "maximum": 120, "type": "integer", "description": "Timeout for HTTP requests in seconds", "default": 30 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```