CF Analytics.

Description

Main reason why did I built this is because i always wanted to know how many problems are there, how many problems contains a specific tag, how many contests are there, and how many contests are there for a specific type. So for my curiousity, I scraped the codeforces website.

Author

NIM	Nama
13522066	Nyoman Ganadipa Narayana

Tools / Library Used

• MariaDB
• Next.js
• TailwindCSS
• shadcn/ui
• recharts
• gocolly

Data Scraping

How To Run

1. Change dir into "/Data Scraping/src"

2. Scrapes the data from codeforces.com

go run main.go

3. While the data isn't complete (you can tell by config.json is not {}), do step 2, otherwise go step 4.

4. The complete data that we need is in ../data/data.json

Scraped Data

The JSON format of the scraped data is

{
    problems: ...
    contests: ...
}

• Inside the problems attribute there contains thousands of problems with each have its own id, title, url, tags, and rating.
• Inside the contests attribute there contains thousands of contests with each have its own id, name, writers, starttime, duration, and the number of participants.

Data Modeling

Here is the ERD design for the database. Where Contributor table is stored is the contest writers, Problem table is stored the problems, Contest table is stored the contests, and Tag table is stored all kind of tags that a problem can have.

ERD DESIGN

Relational Database Table Design

How did i convert the ERD concept into the Relational database design?

• I converted all many to many relationships into a table that contains all the corresponding primary key. (ContributorContest and ProblemTag table)
• I handled all one to many relationship by providing the primary key of the "many" side into the "one", with that "one" side now also has "many" primary keys as its primary keys.

Data Storing

Below is the proof that i had stored the scraped data into the dumped sql.

Contest Table

Contest Table

Contributor Table

Contributor Table

ContributorContest Table

ContributorContest Table

Problem Table

Problem Table

ProblemTag Table

ProblemTag Table

Tag Table

Tag Table

Data Visualization

How to run

1. cd "Data Visualization"/src
2. pnpm install && pnpm dev

Special Thanks

Special thanks to Codeforces for providing the information. codeforces.com