Skip to content

Improve wording and text formatting on "What's in Your Prediction.txt" #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Improve wording and text formatting on "What's in Your Prediction.txt" #4

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 44 additions & 43 deletions What's In Your Prediction.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
#
# What's in *your* prediction?
#
# Every Featrix prediction returns is a JSON structure that includes:
# Every prediction Featrix returns is a JSON object that includes:
#
# 1. An ID to update Featrix on the ground truth if you discover the truth later.
# 2. An echo of your query to verify the query is what you expect.
Expand All @@ -19,68 +19,69 @@
# mixed with Featrix embeddings.

{
prediction_featrix_id: "....", # call a feedback API later with
# the ground truth if you'd like
prediction_time: "Thu Sep 26 18:17:52 2024", # the time the query ran.
prediction_featrix_id: "....", # - Call a feedback API later with
# the ground truth if you'd like.

prediction_time: "Thu Sep 26 18:17:52 2024", # - The time the query ran.

original_query: { # your original input query
numerical_input_1: 2, # dictionary of column: <value> pairs
original_query: { # - Your original input query.
numerical_input_1: 2, # - Dictionary of <column>: <value> pairs.
},

actual_query: { # if you specified a column not
... # present in the model, your query may
}, # be reduced before execution.
actual_query: { # - If you specified a column not
... # present in the model, your query may
}, # be reduced before execution.

ignored_query_columns: [ # for emphasis, we let you know
... # which parts of the original query
], # were ignored.
ignored_query_columns: [ # - For emphasis, we let you know
... # which parts of the original query
], # were ignored.

results: { # your actual prediction: classifier
cat_value_1: "0.25", # with probability or a regression.
results: { # - Your actual prediction: classifier
cat_value_1: "0.25", # with probability or a regression.
...
},

column_guardrails: [ # notes if any column in the query was
{ numerical_input_1: "within 1 stdev" }, # pushing the limit of the training data
query_column_guardrails: [ # - Notes if any column in the query was
{ numerical_input_1: "within 1 stdev" }, # pushing the limit of the training data.
{ numerical_input_2: ">3 stdevs" },
...
],

cross_column_guardrails_score: [ # a cross-column score for how far out
0 to 1 # of training the query was, when considered
], # against all the columns queried.
query_cross_column_guardrails_score: [ # - A cross-column score for how far out
0 to 1 # of training the query was, when considered
], # against all the columns queried.

nearest_neighbors_in_training_data: [ # if security policy allows, we return
{ ...row 234 }, # nearest neighbors in the training space
{ ...row 645 }, ... # for human evaluation of the result.
nearest_neighbors_in_training_data: [ # - If your security policy allows, we return
{ ...row 234 }, # nearest neighbors in the training space
{ ...row 645 }, ... # for human evaluation of the result.
],

upstream_embedding_models: { # metadata of upstream models used.
col1: "sentence-all-MiniLM-L12-v1", # these might be other models you made
col6: "featrix-upstream-52b43b29", # with Featrix or embedding models
col12: "openai-text-embedding-3-large" # from third parties.
upstream_embedding_models: { # - Metadata of upstream models used.
col1: "sentence-all-MiniLM-L12-v1", # These might be other models you made
col6: "featrix-upstream-52b43b29", # with Featrix or embedding models
col12: "openai-text-embedding-3-large" # from third parties.
}

featrix_model: { # the neural function (model)
model_featrix_id: "...", # the Featrix id for this model
model_hash: "...", # the hash to identify a specific artifact
# in the Featrix system that ran this prediction
model_train_time: "...", # when this model was last trained
model_train_loops: 500, # how many passes were made over the data
model_train_featrix_version: "1.2", # the Featrix version that trained this model.
model_predict_featrix_version: "1.2", # the Featrix version that ran this model.
model_metrics: { # metrics from the validation set on this model
f1: 0.75 # MSE, loss, accuracy, precision, recall, et al.
featrix_model: { # - The neural function (model) details.
model_featrix_id: "...", # - The Featrix id for this model.
model_hash: "...", # - The hash to identify a specific artifact
# in the Featrix system that ran this prediction.
model_train_time: "...", # - When this model was last trained.
model_train_loops: 500, # - How many passes were made over the data.
model_train_featrix_version: "1.2", # - The Featrix version that trained this model.
model_predict_featrix_version: "1.2", # - The Featrix version that ran this model.
model_metrics: { # - Metrics from the validation set on this model:
f1: 0.75 # MSE, loss, accuracy, precision, recall, et al.
}
},

featrix_embedding_space: { # the Featrix embedding space information
embedding_space_featrix_id: "...", # the Featrix id for this embedding space
embedding_space_hash: "...", # the hash to identify a specific artifact
embedding_space_train_time: "...", # when this model was last trained
embedding_space_train_loops: 500, # how many passes were made over the data
embedding_space_featrix_version: "1.2", # the Featrix version that trained this model.
embedding_space_run_featrix_version: "1.2", # the Featrix version that ran this model.
featrix_embedding_space: { # - The Featrix embedding space details.
embedding_space_featrix_id: "...", # - The Featrix id for this embedding space.
embedding_space_hash: "...", # - The hash to identify a specific artifact.
embedding_space_train_time: "...", # - When this model was last trained.
embedding_space_train_loops: 500, # - How many passes were made over the data.
embedding_space_featrix_version: "1.2", # - The Featrix version that trained this model.
embedding_space_run_featrix_version: "1.2", # - The Featrix version that ran this model.
}
}

Expand Down