Product SalesChannel visibility bug

[keeskuijpers]

The following bug also occurred in earlier versions.
Within ergonode, we have segments linked to saleschannels.

randomly, this does not always seem to go well. Then suddenly all products are removed one by one from a saleschannel to be added again later.
What is striking is that this only happens with linked ergonode saleschannels.

We have not yet been able to discover what triggers this and since this seems to happen @Randomly this is very undesirable and causes loss of sales and customers.

[keeskuijpers]

Just now it happened again in our shop. We have multiple saleschannels on 1 shopware backend.
The 3 linked saleschannels via segments were disconnected from the products again.

By killing the ProductVisibilitySync process we prevent the shop from being completely empty.

This is a pretty urgent bug!

[thijsMemoICT]

@keeskuijpers Do you have all the products assigned to all saleschannels or are there some that are only two certain saleschannels assigned?

How are the settings of the plugin set?

We do need this kind of information to try to replicate the issue so we can debug it.

[keeskuijpers]

Hi,
I understand what you are saying, this seems to be an old bug that we have been reporting for well over 3/4 years at Ergonode itself.
That's why we are pretty focused on this.
Within shopware we have default setting to a segment which contains all products. The underlying configuration per sales channel then contains a segment with the spcific products per sales channel. There are only a few differences here and there is definitely overlap in the assortment. (90%)
What is striking is that the saleschannels that are not linked to another segment remain active and maintain the product relationship. So the default configuration does not experience this problem.

[thijsMemoICT]

@keeskuijpers could you share how you have done those settings in the plugin config? (Please make sure that data that you do not want to be public is redacted)

My main focus point now is that ProductVisibilityTransformer, here is determined which visibilities should be deleted and which should be added to the product.

Does this problem always exists or at random?

[keeskuijpers]

Hi,

the problem occurs occasionally, 2 - 4 times a week with no apparent cause.

The segments are linked by the api key of the respective segment. From the plugin configuration we choose the saleschannel from the dropdown and place the api key.

The default extension config has the apikey which contains all available products for the shop.

[thijsMemoICT]

@keeskuijpers I have created the same situation on our test instance, and debugged the visibility sync.

There is not a clear cause of this happening.
The following is how the plugin handles the visibility it get's from Ergonode.

• The plugin creates for every SalesChannel in Shopware a GraphQL Client.
• After this for every SalesChannel the products are retrieved from Ergonode, this is done by using the API keys that are associated to the SalesChannel.
• Then the data is processed into an multidimensional array with the SKU of the product as the key and the SalesChannelIds as the values.
• After this step, the data is going to the ProductVisibilityPersistor, this is done on a sku by sku base. Here a ProductVisibilityDTO is created with the SKU and the SalesChannelIds that were in the array before.
• Then it's going into the ProductVisibilityTransformer, here is the logic to which SalesChannels should be removed from the visibility and which ones should be created. In this step, the following is done:

1. Check if there are current visibilities on the product, if there are not, then create a visibility for all SalesChannelIds present in the ProductVisibilityDTO.
2. If there are already visibilities then check if they corespondent with the current visibilities on the product. If it doesn't already exist, create it, if it's not present anymore in the ProductVisibilityDTO then add it to an array for removal.

So the place for it going wrong would be in the ProductVisibilitySyncProcessor where the multidimensional array is constructed, that there the SalesChannelId is not filled for that SKU. Or the only other place would be in the ProductVisibilityTransformer who creates the data that is being used for upserting or deleting. But that last data is based on the multidimensional array and if there would be a logic fault, it should happen more often and on more installations.

The only way I am now seeing to get to the bottom of this and find out what is happening is to literally debug this and write the incoming data to a log file and also the out coming data of the ProductVisibilityTransformer and compare those when the problem is happening.

[keeskuijpers]

Hi,

Thanks for your detailed explanation to investigate this problem.

For your information in our shop it happened 2x today. By killing the php process we stop the disconnection of the sku's.

If we don't do this the process removes per sku the link to the 3 saleschannels with their own api key. All sku's are removed as if there is no saleschannel id.

The remaining saleschannels associated with the default configuration do remain populated.

I have no insight into the structure of the process but this seems quite illogical. That the default configuration does not disconnect anything at the remaining saleschannels.

[thijsMemoICT]

@keeskuijpers I do get that it's not logical and we also think it's quite strange, but we have taken and tested the code that is part of the ProductVisibilitySync and can't see any bugs with the data we are getting from our Ergonode instance.

The only way to figure out what is happening we can think of for now is logging the data that is being retrieved from Ergonode and the ProductVisibilityDTO that are generated from that data. Then we know in which direction to search further if that is data based or code based.

Would it be possible for you to log that data?

If so, then the following would be nice to have logged:

• the $gqlClientGenerator at custom/plugins/ErgonodeIntegrationShopware/src/Processor/ProductVisibilitySyncProcessor.php:40 loggin this can guarantee that all SalesChannels are being processed and not just a subset of them;
• The $skuSalesChannelsMap at custom/plugins/ErgonodeIntegrationShopware/src/Processor/ProductVisibilitySyncProcessor.php:55 this so we know which data is going to the persistor before being transformed to a DTO;
• The $dto at custom/plugins/ErgonodeIntegrationShopware/src/Persistor/ProductVisibilityPersistor.php:31 this so we know if the transform is going alright and to see which data is being generated in the transform and which id's of the visibilities need to be removed.

[keeskuijpers]

We have implemented the requested logging.

[keeskuijpers]

Just now it happened again, herewith the log. I have masked the ids for display here. This is from 1 sku, in reality all sku's were so deleted again

Product Visibility DTO: Ergonode\IntegrationShopware\DTO\ProductVisibilityDTO Object
(
[sku:Ergonode\IntegrationShopware\DTO\ProductVisibilityDTO:private] => A5010130
[salesChannelIds:Ergonode\IntegrationShopware\DTO\ProductVisibilityDTO:private] => Array
(
[0] => 018b6d2208xs7fbbb97c8d91506d26e7
[1] => 018b70d8220e7e8fa12q856ed82fea71
[2] => 018b70d964a77fb1aea6023ff4c5603b
[3] => 018c8e04de927262b0xsdf85f9a1df8d
)

[createPayload:Ergonode\IntegrationShopware\DTO\ProductVisibilityDTO:private] => Array
    (
        [0] => Array
            (
                [visibility] => 30
                [productId] => 018baf599ae970b69ee429bccceed34d
                [salesChannelId] => 018c8e04de927262b0xsdf85f9a1df8d
            )

    )

[deletePayload:Ergonode\IntegrationShopware\DTO\ProductVisibilityDTO:private] => Array
    (
        [0] => Array
            (
                [id] => 0191aa473c67732589bdbae515b5a77b
            )

        [1] => Array
            (
                [id] => 01900d2142d9710b887d39ef1459b496
            )

        [2] => Array
            (
                [id] => 01900d2142d9710b887d39ef15580d50
            )

        [3] => Array
            (
                [id] => 0191aa473c67732589bdbae5164e4768
            )

        [4] => Array
            (
                [id] => 0191aa473c67732589bdbae5170fec61
            )

        [5] => Array
            (
                [id] => 0191aa473c67732589bdbae5172e4b22
            )

        [6] => Array
            (
                [id] => 01900d2142d9710b887d39ef18fe923a
            )

        [7] => Array
            (
                [id] => 01900c963aae7366ad50824b6113b4f6
            )

        [8] => Array
            (
                [id] => 0191a70bc9837204bcd777193bb197d1
            )

        [9] => Array
            (
                [id] => 01919de4d7ac72ef87c5ae5d16e6ce39
            )

        [10] => Array
            (
                [id] => 01900c963ab3710bb1fac8cb41de2511
            )

    )

)

[thijsMemoICT]

@keeskuijpers as I can see from the above I do see that the product should have only 4 assigned SalesChannels, is this correct? And one SalesChannel association is newly added.

Did you also have a log of the $gqlClientGenerator? This way we can check if it goes through all SalesChannels that are available in Shopware. That is the only possible thing now that or the data from Ergonode isn't correct or that not for every SalesChannel the products are retrieved from Ergonode.

[keeskuijpers]

@thijsMemoICT
Hi,
No there are no saleschannels newly added to the product. It does this deleting with each product in the saleschannel in 1 long sync.
This happened yesterday morning, yesterday evening and again this morning.
And after that it can be quiet again for several days.
GQL Client Generator: Generator Object
(
)

[thijsMemoICT]

@keeskuijpers was this the first time it got this product in the sync? As I read the data in the log, it says it has added a SalesChannel to the product.

Maybe we can take a look together outside of Github to the issue in a videocall to hopefull fix this a bit faster.