Performance question

[randyh0329]

I am not sure is there any settings I am missing or configuration I could set up.

I have following test table:

mysql> desc test;
+--------------+------------+------+-----+---------------------+-------+
| Field        | Type       | Null | Key | Default             | Extra |
+--------------+------------+------+-----+---------------------+-------+
| pid          | bigint(21) | YES  |     | NULL                |       |
| id           | bigint(20) | NO   |     | 0                   |       |
| post.id      | bigint(20) | NO   |     | 0                   |       |
| page_id      | bigint(20) | NO   |     | 0                   |       |
| fb_id        | bigint(20) | YES  |     | NULL                |       |
| message      | text       | YES  |     | NULL                |       |
| can_remove   | tinyint(1) | YES  |     | NULL                |       |
| created_time | timestamp  | NO   |     | 0000-00-00 00:00:00 |       |
+--------------+------------+------+-----+---------------------+-------+
8 rows in set (0.19 sec)

replicator_manager.settings tables:
query : SELECT * from test ;
interval : 120

mysql_replicator_multi: execution finished. :setting_name=>cascadedata.test :rows_count=>2174805 :elapsed_time=>7529.38 sec

Any idea? Thanks for any comment. :)

[y-ken]

Hi @randyh0329
Thank you for trying this plugin.
The table of 2 millions rows are quite big!
It seems not too slow for me with scanning 288 rows per seconds.

Would you show me the schema of the table with show create table test;?
If that table hasn't primary key, It need to add that to speed up.

If you could add updated timestamp column, you can use another plugin to collect records of recently updated.
https://github.com/fluent/fluent-plugin-sql#input-how-it-works

[randyh0329]

Hi @y-ken
Sorry. The table already gone.
I will try add primary key at next time and update the performance later.

unfortunately, I can't add timestamp column at this point. otherwise, I will try that plugin.