Bug in training.estimate_parameters_using_expectation_maximisation

[Manuframil]

Hi,

I have a workflow to run the unification of persons running daily. Because is a small sample, about 1000 people, I can train it everyday without a problem.

Everything was working fine until today. Same code, same data schema, but I've got this error:
ValueError: Expected sql condition to refer to one column but got []

I was using splink 4.0.0 when I got the error. Updated to 4.0.8 and still got it.
Checked for missing cols (all datasets have the same schema), missing values, and nothing.

Please, if someone has bump into this error before, help me, because I have no clue.

The code is here:

db_api = DuckDBAPI()

settings = SettingsCreator(
    link_type="link_and_dedupe",
    comparisons= [
        cl.ExactMatch('team_name'),
        comparison_first_name,
        cl.ExactMatch('last_name')
    ],
    blocking_rules_to_generate_predictions=blocking_rules,
    retain_matching_columns=True,
    retain_intermediate_calculation_columns=True,
    em_convergence=0.01
)

linker = Linker(
    [catapult_names, vald_names, antro_names, hidesport_names, lesionados_names, catapult_gk_names, vitruve_names],
    settings, 
    input_table_aliases=["catapult", "vald", "antro", 'hidesport','lesionados', 'catapult_gk', "vitruve"],
    db_api=db_api)

linker.training.estimate_probability_two_random_records_match(blocking_rules, recall=0.7)
linker.training.estimate_u_using_random_sampling(max_pairs=5e5)
training_session_1 = linker.training.estimate_parameters_using_expectation_maximisation(block_on("first_name"))
training_session_2 = linker.training.estimate_parameters_using_expectation_maximisation(block_on("last_name"))
training_session_3 = linker.training.estimate_parameters_using_expectation_maximisation(block_on("team_name"))

And the full error message is here:

[pascal-g-shift]

Hello, I ran into exactly the same error today. I am running on Spark and that code was running yesterday. I did not change the version which is 4.0.8. So it seems that the only thing that could have changed in that environment is the spark backend itself.
The sample code I had tested on splink_datasets.fake_1000 a while ago now fails with the same error.

[Manuframil]

Thanks!! This fix worked for me.

[julianmetricmavens]

For me as well.

[srihari-cdph]

Thank you, %pip install sqlglot==23.0.4
Downgrading sqlglot solved the issue worked for me!

[XzisT33]

Thanks man.. That really works. Had been on this issue for the last 6 days. Had tried even vibe tools but man any day go with old methodologies(read docs/github issues).

[ADBond]

Splink 4.0.9 is now released, which should allow you to unpin sqlglot (but do report if there are still issues!)

[julianmetricmavens]

Hi, I have exactly the same issue since yesterday. We also did not change anything to the input dataset and no update of packages etc.

[jericksonclinicaloptions]

As stated above, there is an issue with the dependent package sqlglot. I believe the last working version was sqlglot==27.14.0, and the Sept 27 release 27.15.0 broke in Splink. (I spent the better part of yesterday troubleshooting this issue). So I pinned to 27.14.0.

FYI: That's a lot of releases

[ADBond]

Thanks for narrowing down the version, very helpful!

[srihari-cdph]

[heart] Sri Hari Sivashanmugam reacted to your message:

…

________________________________ From: XzisT ***@***.***> Sent: Wednesday, September 24, 2025 8:41:52 AM To: moj-analytical-services/splink ***@***.***> Cc: Sri Hari Sivashanmugam ***@***.***>; Comment ***@***.***> Subject: Re: [moj-analytical-services/splink] Bug in training.estimate_parameters_using_expectation_maximisation (Discussion #2771) [Warning: External email] Thanks man.. That really works. Had been on this issue for the last 6 days. Had tried even vibe tools but man any day go with old methodologies(read docs/github issues). — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/moj-analytical-services/splink/discussions/2771*discussioncomment-14496085__;Iw!!B24N9PvjPQId!fAUOXBhgE6gMGDi9K8B31-izHs-qZcg5upE5HBMA6FQJ4vQG5ywT5p-IKVarvm0IsJNoSiBfhib5DXjNsDFKVbq4a7DkpBFGpNp01u7t$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/BI3MBOD3T24G6XO7ZBSCQ5D3UJKNBAVCNFSM6AAAAACG4A224WVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTINBZGYYDQNI__;!!B24N9PvjPQId!fAUOXBhgE6gMGDi9K8B31-izHs-qZcg5upE5HBMA6FQJ4vQG5ywT5p-IKVarvm0IsJNoSiBfhib5DXjNsDFKVbq4a7DkpBFGpBs1C2J6$>. You are receiving this because you commented.Message ID: ***@***.***>

________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail (or the person responsible for delivering this document to the intended recipient), you are hereby notified that any dissemination, distribution, printing or copying of this e-mail, and any attachment thereto, is strictly prohibited. If you have received this e-mail in error, please respond to the individual sending the message, and permanently delete the original and any copy of any e-mail and printout thereof.