Closed
Description
This is an attempt to organize myself and make what I plan to work on more visible
Weekly High Level Goals
- Help @xudong963 to get Release DataFusion
46.0.0
#14123 ready (mostly testing / herding bugs) - Complete a sql-parser-rs release Release sqlparser-rs version
0.55.0
datafusion-sqlparser-rs#1671 - object_store releaase: Release object_store
0.12.0
(API breaking) Around Feb 30 2025 arrow-rs-object-store#28 - write a new "this week in datafusion": March 4, 2025: This week(s) in DataFusion #15005
(BTW if anyone wants to help with writing or releases that would be 🤗 )
Other projects I think are strategically important
These are the ones I plan to look at / review PRs for first
- Anything performance related (though really I am interested in things that help many/all DataFusion users)
- Productionizing / hardening external sorts: A complete solution for stable and safe sort with spill #14692 with @2010YOUY01
- Pretty tree explain plans [EPIC] Complete
SQL EXPLAIN
Tree Rendering #14914 (thanks to @irenjj for starting us off!) - Splitting out
datasources
with @AdamGS and @logan-keede [Epic] Split datasources out fromdatafusion
crate (datafusion/core
) #14444 - pushing expressions down: Support Push down expression evaluation in
TableProviders
#14993
Weekly Stretch / Nice to have goals
- Figure out next steps for Document DataFusion Threading / tokio runtimes (how to separate IO and CPU bound work) #12393
- Figure out where next to improve our performance
- Try to move an example or two from datafusion-examples
Background
I am putting this list on github because:
- I like how github renders checklists w/ PR titles so it is easy to track (I currently have a local text file...)
- I thought others might be interested from seeing what I am doing / planning to do
- It makes me feel better that I don't have time to review all the PRs 😭
The way I prioritize PRs is in the following order
- Bug fixes
- Documentation / UX / API improvements (things that make DataFusion easier/better to work with)
- Performance improvements
- New features with wide appeal
- New functions
Note new features and functions are deliberately at the bottom
Metadata
Metadata
Assignees
Labels
No labels