Delta migration

[JWittmeyer]

PRs:

• Delta migration #303
• Adds delta count methods refinery-submodule-model#169
• Adds delta logic refinery-embedder#144
• https://github.com/code-kern-ai/cognition-task-master/pull/23

Notes:

Threshold is currently a constant 50% of records in embedding so for 100 records > 50 new (accumilated) trigger a full recalculation. This has nothing to do with attribute calc delta. Attributes always calculate only for the delta

Affected Areas:

• embedding creation (For threshold in DB)
• uploading records to refinery (calculates delta & depending on threshold only adds new or recrates the whole)

Added Tests:

• test_upload_records_to_project //add records to a project (not actually upload via s3)
• test_create_embedding //test to create an embedding
• test_update_records_to_project //test for adding & updating records + delta of embedding