Posts

Data Scientist Job Market Research (2026)

Data Scientist Job Market Research (2026) - by my first agent Compiled: 2026-08-06 1. Data Scientist Job Requirements: 2026 vs. 2025 — AI Knowledge on the Rise Clear, sharp increase in AI/GenAI requirements. Core skill shifts (2025 → 2026) SQL remains the #1 most-demanded skill both years — unchanged. Communication moved up to #2, now ranked above Python — companies increasingly want data scientists who can explain findings, not just build models. Python slipped from #2 to #3 in ranking (still essential, just relatively less differentiating). ML skills required in 77% of postings in 2026. The big AI-knowledge jump NLP demand : 5% of postings (2024) → 19% (2025) → NLP mentions up 155% year-over-year — one of the fastest-growing technical asks. Generative AI / LLMs / prompt engineering : mentioned in 31% of data scientist postings in 2026 , up from effectively 0% in 2023 . This is the single starkest change in the market. MLOps now appears in ~8% of postings, reflecting demand fo...
Post a Comment
Read more

My first agent

Intro I've been learning to use Claude Code for data science projects, and my first real test was building a simple agent from scratch — starting with a CLAUDE.md file that defines how the agent should behave. Along the way I hit a classic gotcha (files quietly saving to a temp folder instead of my project directory), fixed it, and then put the whole setup through a real research task. Here's what I learned. Building a Simple Agent with Claude Code (VS Code) The prompt is saved in a CLAUDE.md file, with the following content: # try to build an agent ## Project Context This is an AI agent playground. ## About me I'm a DS, I'm learning to use Claude Code for DS projects. ## Rules - Always ask clarifying questions before starting a complex task - Show your plan and steps before execution - Keep reports and summaries concise — bullet points over paragraphs - Save all reports to the output folder - Cite sources when doing research ## Project Structure - `resources/` -- r...
Post a Comment
Read more

The Time My AI Model Looked 90% Accurate and Was Actually 12%

I built a six-stage machine learning pipeline on 400,000+ consumer complaints from the CFPB (the federal agency that handles complaints about banks, credit bureaus, and lenders)— cleaning raw data, discovering hidden topics, extracting structured facts from free text, predicting outcomes, retrieving similar past cases, and finally, building an AI assistant that triages new complaints using everything the earlier stages learned. The part of this project I'm proudest of isn't any of that. It's the moment I caught myself being wrong. The setup The last stage of the pipeline is a triage tool. Feed it a new complaint, and it pulls up similar historical complaints, then generates a predicted category, a suggested next step, and a confidence level. To know whether an automated prompt-optimization step actually improved this tool, I needed a way to score its predictions against the true category. I wrote what looked like a reasonable check: does a meaningful word from the correct c...
Post a Comment
Read more

My Georgia Tech OMSA Final Year Takeaways (Spring & Summer 2026)

By the final year of OMSA, I could clearly see how the curriculum was coming together. Spring focused on broadening my analytics toolkit—from visual analytics to classical Natural Language Processing—while Summer shifted toward modern Generative AI and an industry-sponsored practicum. Looking back, these four courses complemented one another surprisingly well, bridging traditional analytics, modern AI, and real-world data science practice. CSE 6242 – Data & Visual Analytics CSE 6242 is one of the most practical and technically diverse courses in OMSA. Rather than concentrating on a single subject, the course introduces a wide range of technologies commonly used in data analytics, including D3.js, Tableau, Spark, Docker, SQL, and machine learning workflows. Every homework felt like learning a new technology stack, making the course both exciting and challenging. The workload was among the heaviest in the program. Much of the effort was spent configuring environments, debugging dep...
Post a Comment
Read more

Understanding Customer Behavior Beyond Individual Transactions

Image
Why I Chose This Project This project started with a very personal frustration. During a family trip to London, I used my credit card to tap in and out of the Tube. The first few rides worked perfectly—fast, cashless, effortless. Then, suddenly, my card started getting declined for very small transit charges. There was no lack of credit and nothing that felt suspicious from my point of view, just normal travel behavior. From the system’s perspective, though, my behavior had changed: a new country, repeated small transactions, and a shift away from my usual spending pattern. That gap between how people live and how systems interpret behavior became the motivation for this project. The core question I wanted to explore was simple: Can customer behavior be understood more meaningfully through patterns, rather than individual transactions or rigid rules? The Approaches I Chose (and Why) Before looking at results, it’s important to explain the approaches I intentionally selected , becau...
Post a Comment
Read more

Recreating a Classic Result: ISOMAP, Eigenfaces, and a Simple Face Recognition Experiment

Image
An attempt to reproduce, explore, and truly understand a few classic ideas in dimensionality reduction and face analysis—by doing them myself. Some ideas in data science are so influential that you keep encountering them—in papers, lectures, and references—but they can still feel abstract until you actually work through them yourself. This project was originally presented in a high-impact Science paper, and then extend that exploration to related ideas like eigenfaces and a very simple form of face recognition . Rather than focusing on implementation details, this post is about what I observed , what started to make sense visually , and why these ideas suddenly felt intuitive once I saw the results . Seeing High-Dimensional Data Differently Face images are deceptively complex. A single grayscale image can be represented as thousands of numbers—one per pixel. When you stack hundreds of such images together, the resulting space is extremely high-dimensional, and human intuition stru...
Post a Comment
Read more

Image Compression with Clustering: What I learned

Image
  TL;DR: I used K-means to shrink images by grouping similar colors, and it turns out you don’t need many colors to keep an image looking good (and some choices run a lot faster than others). This post focuses on intuition and observations rather than implementation details, out of respect for academic integrity and future students. Why This Was Interesting In one of my Computational Data Analysis (CDA) homework assignments, I explored how a classic idea from data science:  clustering  can be used for image compression . A digital color image is made up of millions of pixels , and each pixel has three numbers describing its Red, Green, and Blue ( RGB ) values. The key idea behind this homework is simple: Instead of storing every exact color , we can group similar colors together and represent them with a smaller set of “representative” colors. This is exactly what K-means clustering does. Each pixel is treated as a point in color space Similar colors are group...
Post a Comment
Read more