Skip to main content

Overview

The EDL Pipeline produces a unified JSON file (all_stocks_fundamental_analysis.json.gz) containing 86 fields per stock across 2,775 stocks. This schema is the result of combining data from 12+ endpoints and processing through 4 pipeline phases. Output File:
  • Compressed: all_stocks_fundamental_analysis.json.gz (~8-10 MB)
  • Uncompressed: all_stocks_fundamental_analysis.json (~40 MB)
  • Records: 2,775 stocks
  • Fields per stock: 86

Schema Visualization


1. Identity & Classification

Fields (6)

FieldTypeSourceExample
Symbolstringdhan_data_response.json"RELIANCE"
Namestringfundamental_data.json"Reliance Industries Ltd"
Listing Datestringnse_equity_list.csv"29-Nov-1977"
Basic Industrystringfundamental_data.json (CV.INDUSTRY_NAME)"Refineries"
Sectorstringfundamental_data.json (CV.SECTOR)"Energy"
Indexstringdhan_data_response.json (idxlist)"NIFTY 50, NIFTY 100, NIFTY ENERGY"

Index Membership

The Index field contains comma-separated index names. The pipeline tracks 37 major indices:
"Index": "NIFTY 50, NIFTY 100, NIFTY ENERGY"
Tracked Index IDs: 13, 51, 38, 17, 18, 19, 20, 37, 1, 442, 443, 22, 5, 3, 444, 7, 14, 25, 27, 28, 447, 35, 41, 46, 44, 16, 43, 42, 45, 39, 466, 34, 32, 15, 33, 31, 30, 29

2. Fundamentals (Quarterly)

Fields (45)

Latest Quarter

FieldTypeCalculationExample
Latest Quarterstringcq.YEAR (split pipe, index 0)"Q3-FY24"
Net Profit Latest Quarterfloatcq.NET_PROFIT (index 0)18500.0
Net Profit Previous Quarterfloatcq.NET_PROFIT (index 1)17200.0
Net Profit 2 Quarters Backfloatcq.NET_PROFIT (index 2)16800.0
Net Profit 3 Quarters Backfloatcq.NET_PROFIT (index 3)15900.0
Net Profit Last Year Quarterfloatcq.NET_PROFIT (index 4)17800.0
QoQ % Net Profit Latestfloat((Latest - Previous) /Previous) * 1007.56
YoY % Net Profit Latestfloat((Latest - Last Year) /Last Year) * 1003.93

EPS (Earnings Per Share)

FieldTypeCalculationExample
EPS Latest Quarterfloatcq.EPS (index 0)27.5
EPS Previous Quarterfloatcq.EPS (index 1)25.6
EPS 2 Quarters Backfloatcq.EPS (index 2)25.0
EPS 3 Quarters Backfloatcq.EPS (index 3)23.7
EPS Last Year Quarterfloatcq.EPS (index 4)26.5
QoQ % EPS Latestfloat((Latest - Previous) /Previous) * 1007.42
YoY % EPS Latestfloat((Latest - Last Year) /Last Year) * 1003.77
EPS Last Yearfloatcy.EPS (index 0) — Annual98.5
EPS 2 Years Backfloatcy.EPS (index 1) — Annual92.3

Sales (Revenue)

FieldTypeCalculationExample
Sales Latest Quarterfloatcq.SALES (index 0)234000.0
Sales Previous Quarterfloatcq.SALES (index 1)228000.0
Sales 2 Quarters Backfloatcq.SALES (index 2)225000.0
Sales 3 Quarters Backfloatcq.SALES (index 3)220000.0
Sales Last Year Quarterfloatcq.SALES (index 4)232000.0
QoQ % Sales Latestfloat((Latest - Previous) /Previous) * 1002.63
YoY % Sales Latestfloat((Latest - Last Year) /Last Year) * 1000.86
Sales Growth 5 Years(%)float((Current / 5Y Ago)^(1/5) - 1) * 1008.45

OPM (Operating Profit Margin)

FieldTypeCalculationExample
OPM Latest Quarterfloatcq.OPM (index 0)12.5
OPM Previous Quarterfloatcq.OPM (index 1)11.8
OPM 2 Quarters Backfloatcq.OPM (index 2)12.1
OPM 3 Quarters Backfloatcq.OPM (index 3)11.5
OPM Last Year Quarterfloatcq.OPM (index 4)12.3
QoQ % OPM Latestfloat((Latest - Previous) /Previous) * 1005.93
YoY % OPM Latestfloat((Latest - Last Year) /Last Year) * 1001.63
OPM TTM(%)floatttm_cy.OPM12.2
Pipe-Separated Values: All quarterly data from fundamental_data.json comes as pipe-separated strings:
"NET_PROFIT": "18500|17200|16800|15900|17800"
Indexing: 0=Latest, 1=Previous, 2=2Q Back, 3=3Q Back, 4=Same Quarter Last Year

3. Valuation Ratios

Fields (9)

FieldTypeCalculationExample
Market Cap(Cr.)floatCV.MARKET_CAP1658234.5
Stock Price(₹)floatdhan_tech.Ltp2456.75
P/EfloatCV.STOCK_PE24.5
Forward P/EfloatP/E * (TTM EPS / Annualized Current EPS)23.8
Historical P/E 5floatReserved for future use0.0
PEGfloatP/E / YoY EPS Growth6.5
ROE(%)floatroce_roe.ROE14.8
ROCE(%)floatroce_roe.ROCE16.2
D/EfloatNon-Current Liabilities / Total Equity0.29

Calculation Details

Forward P/E

if eps_latest > 0 and pe > 0:
    annualized_eps = eps_latest * 4
    ttm_eps = get_float(ttm_cy.get("EPS"))
    if annualized_eps > 0:
        forward_pe = pe * (ttm_eps / annualized_eps)

PEG Ratio

if yoy_eps > 0 and pe > 0:
    peg = pe / yoy_eps

D/E Ratio

non_current_liab = bs_c.NON_CURRENT_LIABILITIES[0]
total_equity = bs_c.TOTAL_EQUITY[0]
de_ratio = non_current_liab / total_equity if total_equity != 0 else 0.0

4. Ownership & Float

Fields (4)

FieldTypeCalculationExample
FII % change QoQfloatsHp.FII[0] - sHp.FII[1]0.7
DII % change QoQfloatsHp.DII[0] - sHp.DII[1]0.2
Free Float(%)float100 - sHp.PROMOTER[0]49.6
Float Shares(Cr.)float(Market Cap / LTP) * (Free Float / 100)3345.2

Shareholding Pattern Logic

fii_latest = get_value_from_pipe_string(shp.get("FII"), 0)
fii_prev = get_value_from_pipe_string(shp.get("FII"), 1)
fii_change_qoq = fii_latest - fii_prev

promoter_latest = get_value_from_pipe_string(shp.get("PROMOTER"), 0)
free_float_pct = 100.0 - promoter_latest

5. Technical Indicators

Fields (7)

FieldTypeSourceExample
RSI (14)floatdhan_tech.DayRSI14CurrentCandle62.5
SMA Statusstringadvanced_indicators (calculated)"SMA 20: Above (4.9%) | SMA 50: Above (24.1%)"
EMA Statusstringadvanced_indicators (calculated)"EMA 20: Above (6.3%) | EMA 200: Above (72.6%)"
Technical Sentimentstringadvanced_indicators.TechnicalIndicators"RSI: Neutral | MACD: Bearish"
Pivot Pointstringadvanced_indicators.Pivots[0].Classic.PP"245.50"
Gap Up %floatReserved (calculated from intraday data)0.0

SMA/EMA Status Calculation

For each SMA (20, 50, 200):
for s in smas:
    ind_name = s.get("Indicator").replace("-SMA", "")
    val = get_float(s.get("Value"))
    
    if ind_name in ["20", "50", "200"] and val > 0 and ltp > 0:
        diff = ((ltp - val) / val) * 100
        status = "Above" if diff > 0 else "Below"
        sma_signals.append(f"SMA {ind_name}: {status} ({round(diff, 1)}%)")

sma_status = " | ".join(sma_signals)
Output:
"SMA Status": "SMA 20: Above (4.9%) | SMA 50: Below (-2.3%) | SMA 200: Above (15.7%)"

6. Price Performance

Fields (9)

FieldTypeSourceExample
1 Day Returns(%)floatdhan_tech.PPerchange1.23
1 Week Returns(%)floatdhan_tech.PricePerchng1week2.1
1 Month Returns(%)floatdhan_tech.PricePerchng1mon-3.5
3 Month Returns(%)floatdhan_tech.PricePerchng3mon8.2
6 Month Returns(%)floatReserved0.0
1 Year Returns(%)floatdhan_tech.PricePerchng1year15.6
% from 52W Highfloat((LTP - High1Yr) / High1Yr) * 100-12.3
% from 52W LowfloatReserved0.0
% from ATHfloatCalculated from ohlcv_data/*.csv-18.5
Day Range(%)floatReserved0.0
Fields marked as “Reserved” are placeholders for future enhancements. They currently return 0.0.

7. Volume & Liquidity

Fields (6)

FieldTypeSource PhaseExample
RVOLfloatPhase 4 (advanced_metrics_processor.py)1.45
200 Days EMA VolumefloatPhase 4 (advanced_metrics_processor.py)12500000.0
% from 52W High 200 Days EMA VolumefloatPhase 4-15.2
Daily Rupee Turnover 20(Cr.)floatPhase 43450.5
Daily Rupee Turnover 50(Cr.)floatPhase 43280.2
Daily Rupee Turnover 100(Cr.)floatPhase 43120.8
30 Days Average Rupee Volume(Cr.)floatPhase 43350.0

Calculation Logic

RVOL (Relative Volume)

current_volume = today's volume from OHLCV
avg_20_day_volume = mean(last 20 days volume)
RVOL = current_volume / avg_20_day_volume
Interpretation:
  • RVOL > 1.5 — High volume (50% above average)
  • RVOL = 1.0 — Average volume
  • RVOL < 0.5 — Low volume (50% below average)

Daily Rupee Turnover

turnover_20 = mean(price * volume for last 20 days) / 10^7  # Convert to Crores

8. Volatility

Fields (4)

FieldTypeSource PhaseExample
5 Days MA ADR(%)floatPhase 4 (advanced_metrics_processor.py)2.3
14 Days MA ADR(%)floatPhase 42.5
20 Days MA ADR(%)floatPhase 42.4
30 Days MA ADR(%)floatPhase 42.6

ADR (Average Daily Range) Calculation

for each day in period:
    daily_range = ((high - low) / low) * 100

ADR = mean(daily_range for period)
Example:
# Day 1: Low=2400, High=2480 → Range = 3.33%
# Day 2: Low=2440, High=2475 → Range = 1.43%
# ...
# 20 Days MA ADR = mean([3.33, 1.43, ...]) = 2.4%
Interpretation:
  • ADR > 5% — High volatility
  • ADR 2-5% — Normal volatility
  • ADR < 2% — Low volatility

9. Circuit & Price Bands

Fields (1)

FieldTypeSource PhaseExample
Circuit LimitstringPhase 4 (add_corporate_events.py)"20%"

Circuit Limit Values

  • 20% — Standard circuit (most stocks)
  • 10% — Restricted circuit
  • 5% — Highly restricted (ASM/GSM stocks)
  • 2% — Extreme restriction
Source: Matched from complete_price_bands.json (NSE CSV)

10. Earnings Tracking

Fields (3)

FieldTypeSource PhaseExample
Quarterly Results DatestringPhase 4 (process_earnings_performance.py)"03-Mar-2024"
Returns since Earnings(%)floatPhase 48.5
Max Returns since Earnings(%)floatPhase 412.3

Calculation Logic

# Step 1: Find latest results filing from company_filings/*.json
results_filings = [f for f in filings if "Results" in f.caption]
latest_results = results_filings[0]  # Most recent
results_date = latest_results.news_date

# Step 2: Get price from OHLCV on results_date - 1 (pre-earnings close)
pre_earnings_close = ohlcv[results_date - 1].close

# Step 3: Calculate returns
returns_since_earnings = ((current_price - pre_earnings_close) / pre_earnings_close) * 100

# Step 4: Calculate max returns (peak since results date)
max_price_since_results = max(ohlcv[results_date:].high)
max_returns_since_earnings = ((max_price_since_results - pre_earnings_close) / pre_earnings_close) * 100
Example:
  • Results Date: Mar 3, 2024
  • Pre-Earnings Close: ₹2,265
  • Current Price: ₹2,456 → Returns = +8.5%
  • Max Price Since Results: ₹2,543 → Max Returns = +12.3%

11. Event Markers

Field (1)

FieldTypeSource PhaseExample
Event MarkersarrayPhase 4 (add_corporate_events.py)See below

Event Marker Structure

"Event Markers": [
  {"icon": "★", "name": "STASM", "trigger": "Stock in ASM Stage 1"},
  {"icon": "📊", "name": "Results Recently Out", "trigger": "Released 5 days ago"},
  {"icon": "💸", "name": "Dividend (15-Mar)", "trigger": "Ex-date approaching"}
]

Complete Event Icons Reference

IconNameTrigger Condition
LTASM / STASMStock in ASM surveillance groups
📊Results Recently OutResults released in last 7 days
🔑Insider TradingSEBI Reg 7(2) / Form C filing in last 15 days
📦Block DealBulk/Block deal in last 7 days
#+/- RevisionPrice band revision detected (incremental_price_bands.json)
Results (DD-Mon)Upcoming results announcement (with date)
🎁Bonus (DD-Mon)Upcoming bonus issue (with date)
✂️Split (DD-Mon)Upcoming stock split (with date)
💸Dividend (DD-Mon)Upcoming dividend (with ex-date)
📈Rights (DD-Mon)Upcoming rights issue (with date)

Trigger Logic Examples

ASM/GSM Surveillance

if symbol in nse_asm_list:
    stage = nse_asm_list[symbol].surveillance_stage
    event_markers.append({
        "icon": "★",
        "name": stage,  # "STASM" or "LTASM"
        "trigger": f"Stock in {stage}"
    })

Results Recently Out

results_filings = [f for f in filings if "Results" in f.caption]
if results_filings:
    latest_date = parse_date(results_filings[0].news_date)
    days_ago = (today - latest_date).days
    if days_ago <= 7:
        event_markers.append({
            "icon": "📊",
            "name": "Results Recently Out",
            "trigger": f"Released {days_ago} days ago"
        })

Upcoming Dividend

upcoming_actions = [a for a in upcoming_corporate_actions if a.Symbol == symbol]
dividend_actions = [a for a in upcoming_actions if a.ActionType == "Dividend"]
if dividend_actions:
    ex_date = dividend_actions[0].ExDate
    amount = dividend_actions[0].Details  # e.g., "Rs 9 per share"
    event_markers.append({
        "icon": "💸",
        "name": f"Dividend ({ex_date})",
        "trigger": f"{amount}, Ex-date: {ex_date}"
    })

12. Recent Announcements (Regulatory)

Field (1)

FieldTypeSource PhaseExample
Recent AnnouncementsarrayPhase 4 (add_corporate_events.py)See below

Structure

Top 5 most recent regulatory filings:
"Recent Announcements": [
  {
    "Date": "03-Mar-2024",
    "Headline": "Outcome of Board Meeting - Quarterly Results",
    "URL": "https://www.bseindia.com/xml-data/corpfiling/AttachHis/abc123.pdf"
  },
  {
    "Date": "28-Feb-2024",
    "Headline": "Insider Trading - SEBI Reg 7(2)",
    "URL": "https://www.bseindia.com/xml-data/corpfiling/AttachHis/xyz789.pdf"
  },
  {
    "Date": "15-Feb-2024",
    "Headline": "Intimation of Board Meeting",
    "URL": "https://www.bseindia.com/xml-data/corpfiling/AttachHis/def456.pdf"
  }
]

Data Source

Files: company_filings/{SYMBOL}_filings.json Sorting: By news_date descending (newest first) Limit: Top 5 announcements

13. News Feed (Media)

Field (1)

FieldTypeSource PhaseExample
News FeedarrayPhase 4 (add_corporate_events.py)See below

Structure

Top 5 most recent news items with AI sentiment:
"News Feed": [
  {
    "Title": "Reliance Q3 Results Beat Estimates",
    "Sentiment": "positive",
    "Date": "2024-03-03T14:30:00Z"
  },
  {
    "Title": "Jio Subscriber Growth Slows in Q3",
    "Sentiment": "neutral",
    "Date": "2024-03-02T10:15:00Z"
  },
  {
    "Title": "Reliance Retail Plans Aggressive Expansion",
    "Sentiment": "positive",
    "Date": "2024-03-01T16:45:00Z"
  }
]

Data Source

Files: market_news/{SYMBOL}_news.json Sorting: By published_date descending (newest first) Limit: Top 5 news items

Sentiment Values

  • positive — Bullish news (🟢)
  • neutral — Factual reporting (⚪)
  • negative — Bearish news (🔴)

Complete Schema Example

Single Stock Record

{
  "Symbol": "RELIANCE",
  "Name": "Reliance Industries Ltd",
  "Listing Date": "29-Nov-1977",
  "Basic Industry": "Refineries",
  "Sector": "Energy",
  "Index": "NIFTY 50, NIFTY 100, NIFTY ENERGY",
  "Market Cap(Cr.)": 1658234.5,
  "Stock Price(₹)": 2456.75,
  "Latest Quarter": "Q3-FY24",
  "Net Profit Latest Quarter": 18500.0,
  "Net Profit Previous Quarter": 17200.0,
  "Net Profit 2 Quarters Back": 16800.0,
  "Net Profit 3 Quarters Back": 15900.0,
  "Net Profit Last Year Quarter": 17800.0,
  "QoQ % Net Profit Latest": 7.56,
  "YoY % Net Profit Latest": 3.93,
  "EPS Latest Quarter": 27.5,
  "EPS Previous Quarter": 25.6,
  "EPS 2 Quarters Back": 25.0,
  "EPS 3 Quarters Back": 23.7,
  "EPS Last Year Quarter": 26.5,
  "QoQ % EPS Latest": 7.42,
  "YoY % EPS Latest": 3.77,
  "EPS Last Year": 98.5,
  "EPS 2 Years Back": 92.3,
  "Sales Latest Quarter": 234000.0,
  "Sales Previous Quarter": 228000.0,
  "Sales 2 Quarters Back": 225000.0,
  "Sales 3 Quarters Back": 220000.0,
  "Sales Last Year Quarter": 232000.0,
  "QoQ % Sales Latest": 2.63,
  "YoY % Sales Latest": 0.86,
  "Sales Growth 5 Years(%)": 8.45,
  "OPM Latest Quarter": 12.5,
  "OPM Previous Quarter": 11.8,
  "OPM 2 Quarters Back": 12.1,
  "OPM 3 Quarters Back": 11.5,
  "OPM Last Year Quarter": 12.3,
  "QoQ % OPM Latest": 5.93,
  "YoY % OPM Latest": 1.63,
  "OPM TTM(%)": 12.2,
  "P/E": 24.5,
  "Forward P/E": 23.8,
  "Historical P/E 5": 0.0,
  "PEG": 6.5,
  "ROE(%)": 14.8,
  "ROCE(%)": 16.2,
  "D/E": 0.29,
  "FII % change QoQ": 0.7,
  "DII % change QoQ": 0.2,
  "Free Float(%)": 49.6,
  "Float Shares(Cr.)": 3345.2,
  "1 Day Returns(%)": 1.23,
  "1 Week Returns(%)": 2.1,
  "1 Month Returns(%)": -3.5,
  "3 Month Returns(%)": 8.2,
  "6 Month Returns(%)": 0.0,
  "1 Year Returns(%)": 15.6,
  "% from 52W High": -12.3,
  "% from 52W Low": 0.0,
  "% from ATH": -18.5,
  "Gap Up %": 0.0,
  "Day Range(%)": 0.0,
  "RSI (14)": 62.5,
  "SMA Status": "SMA 20: Above (4.9%) | SMA 50: Above (24.1%) | SMA 200: Above (15.7%)",
  "EMA Status": "EMA 20: Above (6.3%) | EMA 50: Above (18.2%) | EMA 200: Above (72.6%)",
  "Technical Sentiment": "RSI: Neutral | MACD: Bullish",
  "Pivot Point": "2456.0",
  "RVOL": 1.45,
  "200 Days EMA Volume": 12500000.0,
  "% from 52W High 200 Days EMA Volume": -15.2,
  "Daily Rupee Turnover 20(Cr.)": 3450.5,
  "Daily Rupee Turnover 50(Cr.)": 3280.2,
  "Daily Rupee Turnover 100(Cr.)": 3120.8,
  "30 Days Average Rupee Volume(Cr.)": 3350.0,
  "5 Days MA ADR(%)": 2.3,
  "14 Days MA ADR(%)": 2.5,
  "20 Days MA ADR(%)": 2.4,
  "30 Days MA ADR(%)": 2.6,
  "Circuit Limit": "20%",
  "Quarterly Results Date": "03-Mar-2024",
  "Returns since Earnings(%)": 8.5,
  "Max Returns since Earnings(%)": 12.3,
  "Event Markers": [
    {"icon": "📊", "name": "Results Recently Out", "trigger": "Released 5 days ago"},
    {"icon": "💸", "name": "Dividend (15-Mar)", "trigger": "Rs 9 per share, Ex-date: 15-Mar-2024"}
  ],
  "Recent Announcements": [
    {
      "Date": "03-Mar-2024",
      "Headline": "Outcome of Board Meeting - Quarterly Results",
      "URL": "https://www.bseindia.com/xml-data/corpfiling/AttachHis/abc123.pdf"
    },
    {
      "Date": "28-Feb-2024",
      "Headline": "Insider Trading - SEBI Reg 7(2)",
      "URL": "https://www.bseindia.com/xml-data/corpfiling/AttachHis/xyz789.pdf"
    }
  ],
  "News Feed": [
    {
      "Title": "Reliance Q3 Results Beat Estimates",
      "Sentiment": "positive",
      "Date": "2024-03-03T14:30:00Z"
    },
    {
      "Title": "Jio Subscriber Growth Slows in Q3",
      "Sentiment": "neutral",
      "Date": "2024-03-02T10:15:00Z"
    }
  ]
}

Field Count Summary

CategoryField Count
Identity & Classification6
Fundamentals (Quarterly)45
Valuation Ratios9
Ownership & Float4
Technical Indicators7
Price Performance9
Volume & Liquidity7
Volatility4
Circuit & Price Bands1
Earnings Tracking3
Event Markers1
Recent Announcements1
News Feed1
Total86

Next Steps

Pipeline Architecture

Learn how these 86 fields are built through the 4-phase pipeline

Data Sources

Explore the 12+ endpoints that feed data into this schema

Build docs developers (and LLMs) love