Participants classify news articles into one of three political bias categories: left, center, or right. This is a supervised document-level classification task where the input is the full article body, not only the headline. Each article receives one political bias label, making the task a three-class, single-label classification problem.

To participate, train and run your own model, generate a prediction.csv file in the correct format, and submit it for evaluation on the held-out test dataset. Valid submissions are scored and placed on the public leaderboard.

The dataset contains 21,747 news articles collected from AllSides balanced news headline roundups in November 2022. AllSides presents expert-selected U.S. news articles from sources with different political perspectives, including left, center, and right viewpoints, to help readers compare coverage and recognize bias.

Each article was tagged by four expert annotators according to expressed political partisanship: left, right, or neutral/center. The collected data includes headlines, dates, article text, topic tags, publishing outlet, and AllSides' neutral topic description. The original scrape was collected on 2022-11-15 from AllSides headline roundups.

1. Clone the starter repository.
2. Create a virtual environment and install dependencies.
3. Train your own model and run it on test.csv.

Train a baseline, a fine-tuned transformer, or any custom architecture as long as it can produce one valid bias label for each article in the test split. The starter kit is intended as a reference workflow, but participants are expected to run their own model choices, tune them on the training and validation data, and generate predictions for final evaluation.

Run inference on the provided test inputs and save the predictions as a CSV. Each row should contain the article text and your predicted label. Labels must be one of left, center, or right.

Name the file prediction.csv before submitting it for leaderboard evaluation.

• File must be named exactly prediction.csv.
• Labels must be one of left, center, or right in lowercase.
• Use one row per test article so every test item receives exactly one prediction.
• Up to 5 submissions per day, unlimited during the warmup period.

Scoring is performed against the held-out test set using Accuracy and Macro F1. The leaderboard ranks by Macro F1, with Accuracy as tiebreaker. Scored submissions appear on the leaderboard after evaluation; failed submissions receive a validation error.