Abel Yagubyan

AI Researcher & Founder

Hello! I'm an AI researcher and founder working on trustworthy AI evaluation: the reliability of LLM-as-a-judge scoring and the reproducibility of LLM agents. I'm a Ph.D. student in Artificial Intelligence at the University of Georgia (since August 2026), and I hold an M.S. in Computer Science from Northwestern University (Summa Cum Laude) and dual B.A. degrees in Computer Science & Applied Mathematics from UC Berkeley.

I'm currently building LearnOS, an open-source, agentic AI university. I contribute to open-source LLM evaluation as an Area Triager on Snowflake's TruLens and a contributor to the UK AI Security Institute's inspect_evals, and I've completed 200+ peer reviews across nine Elsevier AI journals. I was accepted into Y Combinator's Spring 2026 batch. Earlier, I spent two years as a Senior Data Scientist at C3.ai; co-founded FibonAI (UC Berkeley SkyDeck); researched UPC++ performance at Lawrence Berkeley National Laboratory; and interned at Apple.

200+

Peer Reviews

Papers

Citations

Journals

Citing Groups

News

TruLens 2.16.0 ships Jury.repeated, repeated-trial LLM-as-a-judge scoring built on the Coin Flip Judge paper; 30 of my PRs are now in TruLens releases.

Oct 2026

inspect_evals' BEST_PRACTICES.md cites two of my merged PRs as reference examples; credited in releases v0.22.0 and v0.25.0.

Oct 2026

Named Area Triager on TruLens (the open-source LLM and agent evaluation library maintained by Snowflake) and added to MAINTAINERS.md.

Sep 2026

Contribution credited in inspect_evals v0.20.0, the UK AI Security Institute's evaluation suite for Inspect AI.

Sep 2026

"The Coin Flip Judge?" accepted at GroundLM 2026 (EMNLP 2026) as an Archival Long Paper; camera-ready submitted.

Sep 2026

Started a Ph.D. in Artificial Intelligence at the University of Georgia

Aug 2026

Released two sole-author preprints on LLM agent reproducibility and LLM-as-a-judge reliability.

Apr 2026

Promoted to Senior Data Scientist at C3.ai in under 18 months for high-impact customer solutions

2024

Co-founded FibonAI, accepted into UC Berkeley's SkyDeck Pad-13 Incubator competing against 5,000+ startups

June 2023 - Feb 2024

Graduated with M.S. in Computer Science from Northwestern University with Summa Cum Laude Honors

June 2023

Research contributor at Lawrence Berkeley National Lab's Pagoda Project on UPC++ performance testing

June 2022 - April 2023

Graduated from UC Berkeley with dual B.A. in Computer Science & Applied Mathematics

May 2022

Embedding programming IDEs into testing software: illustration of an in-browser IDE, compiler, and autograding pipeline

Presented "Seamless Embedding of Programming IDEs into Computer-Based Testing Software" at ACM SIGCSE '22

March 2022

Software Engineering Intern at Apple x UC Berkeley, managing CS61C projects for 4000+ HBCU students

May - Aug 2021

Research & Publications

The Coin Flip Judge: conceptual illustration of an unreliable coin-flip verdict on an unstable balance scale

GroundLM 2026 (EMNLP), Archival 15 citations

The Coin Flip Judge? Reliability and Bias in LLM-as-a-Judge Evaluation

Abel Yagubyan (sole author)

arXiv:2606.13685, April 2026. Accepted as an Archival Long Paper at the GroundLM 2026 workshop at EMNLP 2026, Budapest (proceedings forthcoming).

TLDR: Repeated identical LLM-as-a-judge evaluations across six judges from three providers show pairwise verdicts flip on 11 to 14 percent of re-runs, order bias persists across model generations, a single trial recovers only 86.6 percent of the consensus verdict, and about 11 trials are needed to stabilize a verdict; argues for multi-trial aggregation, order randomization, and uncertainty reporting as standard practice.

Cited by Meta Superintelligence Labs

arXiv abstract PDF

LLM agent reproducibility: conceptual illustration of one task branching into two divergent tool-call pipelines

Preprint 10 citations

How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines

Abel Yagubyan (sole author)

arXiv:2605.28840, April 2026.

TLDR: Measures whether an LLM agent given the same task twice takes the same actions across several models and providers, and documents that agents with large unconstrained action spaces vary run to run.

Cited by Alibaba

arXiv abstract PDF

UPC++ Performance Regression Testing & Benchmarking

Abel Yagubyan, Lawrence Berkeley National Laboratory

Pagoda Project, Research Contribution

TLDR: Developed automated performance regression testing platform for UPC++, a C++ library for distributed-memory parallel computing. Implemented distributed hashing benchmarks and compared performance against MPI and SHMEM.

LBNL Pagoda

SCALPEL: Customized Deep Neural Network Compression

Abel Yagubyan

Northwestern University, Research Project (Jan 2022 to June 2022)

TLDR: Developed Python-based DNN compression framework that reduced model sizes (AlexNet, LeNet-5) by up to 80% and achieved 3.5x speedup by customizing pruning techniques to target hardware parallelism (CPU, GPU, Microcontroller).

Report GitHub

Lick Observatory Supernova Search: illustration of the supernova photometry pipeline and light curves

MNRAS 2022 19 citations

The Lick Observatory Supernova Search follow-up program: photometry data release of 70 SESNe

WeiKang Zheng, Benjamin E. Stahl, Alexei V. Filippenko, et al. (incl. Abel Yagubyan)

Monthly Notices of the Royal Astronomical Society, vol. 512, pp. 3195-3214 (2022).

TLDR: Public BVRI and unfiltered light curves for 70 stripped-envelope supernovae observed 2003 to 2020, released through an automated photometry pipeline; the data yield host-galaxy extinction estimates and, for the first time, accurate rise-time measurements for a large SESN sample, with SNe Ic showing lower ejecta masses and velocities but higher nickel-56 masses than SNe Ib and IIb.

DOI MNRAS (OUP)

Seamless Embedding of Programming IDEs into Computer-Based Testing Software

Abel Yagubyan, Dan Garcia

ACM SIGCSE '22, Technical Symposium on Computer Science Education (Vol. 2)

TLDR: Presents an interactive RISC-V compiler integrated into PrairieLearn for online programming education, enabling 3,000+ students to complete CS61C exams with embedded IDE functionality.

ACM DL

Industry Experience

C3.ai

Senior Data Scientist

May 2024 - March 2026 | Redwood City, CA

Led data science initiatives in predictive maintenance and generative AI across large-scale deployments for Shell, ExxonMobil, Dow, Bloom Energy. Directed pilots delivering over $10M in value and deploying 1,000+ models per project. Developed foundational healthcare transformer model trained on 1M+ U.S. patient records. Created SpeedyREL, an internal tool reducing deployment timelines by 50%, adopted by 5+ projects. Promoted to Senior in under 18 months.

Python PyTorch Transformers XGBoost SQL Kubernetes

FibonAI

Co-Founder

June 2023 - February 2024 | San Francisco, CA

Designed and built SaaS platform providing comprehensive LLM-powered workspace for in-house corporate legal teams—integrating intake & triage, contract management, matter management, and AI Assistants. Successfully onboarded 8 General Counsels/Chief Legal Officers. Achieved 60% conversion rate during demos and 100% weekly user growth. Accepted into UC Berkeley's SkyDeck Pad-13 Incubator (competing against 5,000+ startups globally).

Python Flask AWS MongoDB GPT-4 React

Apple x UC Berkeley

Software Engineering Intern

May 2021 - August 2021 | Remote

Project lead in collaboration with Apple and UC Berkeley's CS61C: Computer Architecture course. Coded and managed Computer Architecture, Algorithm, and ML-based projects for 4,000+ students simultaneously. Developed implementations using C, C++, Python, SIMD, OpenMP, and RISC-V Assembly.

C C++ Python RISC-V Assembly OpenMP

Software & Projects

SynthoraPlatform

AI development platform that builds full web and mobile apps from natural language: frontend, backend, database, workflows, and deployment.

TypeScript No-code

GitHub

EasyVoiceCloneOpen Source

A minimal, open-source pipeline to clone any voice from just a few audio samples, built for creators and researchers experimenting with AI voice synthesis.

JavaScript Voice AI

GitHub

TableSageAI

Turns any CSV into an instant exploratory-data-analysis and lightweight-modeling workspace: cleaning suggestions, schema understanding, statistical insights, and sketch ML models with interpretable explanations.

Python Data + ML

GitHub

Prompt2DeckAI

Generates presentation decks (PowerPoint, Google Slides, or PDF) from a topic or outline, combining LLM content expansion with automated visuals for presentation-ready slides.

Python Slide gen

GitHub

ArxivScribeTool

A Discord and Slack bot that monitors arXiv, extracts the most relevant new machine-learning papers, and posts clean, LLM-generated TLDR summaries every day.

Python arXiv bot

GitHub

EHRJEPAResearch

A JEPA-based self-supervised framework for learning predictive patient embeddings from structured electronic health record data.

Python Self-supervised

GitHub

Open Source

LearnOS

Creator & Maintainer

Open-source AI university · github.com/Abelo9996/LearnOS · MIT · JavaScript

A self-hosted, open-source "AI university" built as a system of specialized AI agents (curriculum, Socratic tutor, assessment, research, analytics, and profiling) that design personalized learning roadmaps, teach in real time, and grade toward mastery-based certification. Runs on a single OpenRouter key so any major model can power any agent, with SQLite-persisted progress and no accounts or paywall. Built with Node.js and Vite.

GitHub Vision

TruLens

Area Triager

Maintained by Snowflake · github.com/truera/trulens · Area Triager since September 11, 2026

Open-source LLM and agent evaluation and tracing library. 30 merged pull requests (August to October 2026), all shipped in public PyPI releases 2.14.0, 2.15.0 and 2.16.0, and 29 of them with tests. Built Jury.repeated with reliability metadata on every Jury result (#2854), repeated-trial LLM-as-a-judge scoring based on the Coin Flip Judge paper, listed under New features in the 2.16.0 release. The other merged work fixes OpenTelemetry and async instrumentation, the database and persistence layer, the evaluation API, batch evaluation, providers, and the dashboard. Filed 22 issues, each closed by one of these PRs, and posted 16 reviews of other contributors' PRs, including catches of a core-only install crash and of a failing certification check in the 2.16.0 release PR before it shipped. 12th of 124 contributors by commits to main. Listed in MAINTAINERS.md with scope over OpenTelemetry instrumentation, evaluation and database correctness, and shared core code.

MAINTAINERS.md Merged PRs Jury.repeated (#2854) Release 2.16.0

inspect_evals

Contributor

UK AI Security Institute · github.com/UKGovernmentBEIS/inspect_evals

The UK AI Security Institute's evaluation suite for Inspect AI. 4 merged pull requests, credited by handle in releases v0.20.0, v0.22.0 and v0.25.0: moving the per-key metric helpers onto the framework's aggregate() (#2317), moving the unscored-sample signal to the first-class Score.reason field across 9 evals (#2459), and fixing makemesay's four rate metrics, which miscounted games whose repeated trials ended differently (#2547), plus a one-line README fix. The repo's BEST_PRACTICES.md cites #2547 and #2317 as reference examples, and maintainers named #2547 when building a shared test helper for epoch-reduced metrics (#2597, #2641).

Merged PRs BEST_PRACTICES.md Release notes v0.25.0

Awards & Recognition

GroundLM 2026 (EMNLP 2026)

Budapest · 2026

"The Coin Flip Judge?" accepted as an Archival Long Paper.

Peer Reviewer

200+ reviews · 9 journals · Oct 2026

Verified peer reviews across Elsevier AI and computer-vision journals.

ORCID record

Y Combinator

March 2026

Accepted into Y Combinator's Spring 2026 batch.

Summa Cum Laude

Northwestern University · June 2023

Graduated with Summa Cum Laude honors from the M.S. in Computer Science program.

Elsevier Recognised Reviewer Certificates

2024 to 2026 · nine journals, by review volume

Engineering Applications of Artificial Intelligence109
Image and Vision Computing33
Neural Networks26
Information Fusion18
Neurocomputing3
Expert Systems with Applications1
Knowledge-Based Systems1
Data and Knowledge Engineering1
International Journal of Intelligent Networks1