Fix gbak output some errors and warnings to stderr instead of stdout

[XaBbl4]

Currently in standalone application mode when redirect to the standard stream, for example:
gbak ... > /path/to/stdout.log
some error and warning messages may be missed, which may cause inconvenience.

For example, during when restore we may see an error message:

gbak: ERROR:Database is not online due to failure to activate one or more indices.
gbak: ERROR:    Run gfix -online to bring database online without active indices.

but in order to find out which index remains inactive, we need to look at the entire log, and in it we can already find:

gbak:cannot commit index TEST_INDEX
gbak: ERROR:attempt to store duplicate value (visible to active transactions) in unique index "TEST_INDEX"
gbak: ERROR:    Problematic key value is ("ID" = 1)

Although in the global community it is accepted to output all errors and warnings to stderr.

This patch fixes this bug. I tried to cover all the cases found, but may have missed something.
I believe BURP_print_status should always output to stderr, regardless of whether the err argument is set to true or not. The err argument is now only responsible for setting status in service mode. Also BURP_print_warning should always output to stderr.

[AlexPeshkoff]

Earlier the following approach was used - errors that may be recovered by gbak were going to stdout, fatal - to stderr. I can agree that putting recoverable errors into stderr makes sense. Bit why not all? Something like:

         BURP_print(false, 171, relation->rel_name.toQuotedString().c_str());
         // msg 171: error committing metadata for relation %s

in restore.epp is left in stdout but

         BURP_print(true, 343, SafeArg() << int(attribute) << "put_asciz()" << (MAX_FILE_NAME_SIZE - 1));
         // msg 343: text for attribute @1 is too large in @2, truncating to @3 bytes

in backup.epp will go into stderr.

I want to know what criteria was used to keep some messages in stdout.

[XaBbl4]

Initially, during the restore, I tried to capture only those errors that didn't bring the database online when the utility terminated.

As I mentioned in the first message, I might have missed some places, as I couldn't reproduce every case; this is likely one of them.

In the backup, I moved all BURP_print calls to the error stream, not seeing the logic for splitting the output.
As for error 343, the code had a similar output in the put_text function, and put_asciz also sent it to the error stream.

[AlexPeshkoff]

Andrey, may be I was not enough explicit. I suggest to:

1. Complete cleanup process in restore.epp. There is no need to reproduce all bugs - just find all BURP_print calls with false parameter. Regarding previous comments about about errors 171 & 343 - it's completely OK to sent output to stderr for both of them. Like all other invocations of BURP_error.

2. After cleanup finished - may be argument err should be removed from BURP_print() at all? For a few remaining calls in burp.cpp like
   BURP_print(false, 91, FB_VERSION);
   may be invite new function name?

[XaBbl4]

1. Remove err argument from BURP_print and BURP_print_status. All messages from these functions are now printed to standard error stream. In BURP_print_status add serviceFlag argument for set service code to service and don't print to standart streams.

2. BURP_print with output version has been replaced to BURP_message.

3. In additional calls BURP_error_redirect with NULL status argument replaced to BURP_error with abort argument, because calling BURP_print_status only adds small overhead.

4. I have one question about the output of msg 81. There are two outputs in the get_array function, but one had BURP_print_status(true, ...) and the other had BURP_print_status(false, ...). I left it as is for now. According to history, you did this in commit 7403516, and it looks like there was a typo, or maybe I just missed the sense.

[AlexPeshkoff]

4. I have one question about the output of `msg 81`. There are two outputs in the `get_array` function, but one had `BURP_print_status(true, ...)` and the other had `BURP_print_status(false, ...)`. I left it as is for now. According to history, you did this in commit [7403516](https://github.com/FirebirdSQL/firebird/commit/740351638f7cb042fa88c5cf08b7527c0be1c49f), and it looks like there was a typo, or maybe I just missed the sense.

Certainly - it was a bug. Output direction for BURP_print and related BURP_print_status should certainly match. Looks like arrays are used not too often, and errors saving them to disk during restore are even more rare. According to old logic both of them were to go to stdout, according to new one - both to stderr.

[XaBbl4]

Certainly - it was a bug. Output direction for BURP_print and related BURP_print_status should certainly match. Looks like arrays are used not too often, and errors saving them to disk during restore are even more rare. According to old logic both of them were to go to stdout, according to new one - both to stderr.

Changed this to one behavior

[pavel-zotov]

This is example of SQL which creates unrecoverable DB (because of dplicates in index which will appear at restore phase):

set bail on;
set list on;
set echo on;
shell if exist r:\temp\tmp4test.fdb del r:\temp\tmp4test.fdb;
create database 'localhost:r:\temp\tmp4test.fdb' user 'sysdba' password 'masterkey';

create table test(id int);
insert into test select row_number()over() from rdb$types rows 10;
commit;
set term ^; execute block as begin rdb$set_context('USER_SESSION', 'TMP', 1); end ^ set term ;^
create unique index test_x_unq on test computed by ( coalesce(cast(rdb$get_context('USER_SESSION', 'TMP') as int), 0) * id );
commit;

Let's create this DB and make backup.
Then try to restore:

1. using gbak -rep ...
2. using fbsvcmgr action_restore res_replace ...

Doing that on 6.0.0.1803 vs 6.0.0.1850 shows unexpected difference between logs for fbsvcmgr (gbak is OK).
It is better to show problem using following screen:

(i.e. problem-related lines which were shown in STDOUT before this fix now are not displayed at all)
Also, this outcome can be seen in attached .xlsx file (inside .zip)
gh-8793_-unrecoverable_db-_gbak-vs-fbsvcmgr.xlsx.zip

[XaBbl4]

Yes, I've already encountered a problem where warnings and errors aren't being output through services, because the Service::outputError procedure is essentially empty, with only fb_assert(False)
I initially made a commit where service warnings are output to stdout.

But now I'm more inclined to redirect all service error output to standard output, something like:

void Service::outputError(const char* text)
{
    outputVerbose(text);
}

@AlexPeshkoff, what do you think? Will that be enough?

[AlexPeshkoff]

On 3/26/26 11:18, Andrey Kravchenko wrote: *XaBbl4* left a comment (FirebirdSQL/firebird#8793) <#8793?email_source=notifications&email_token=AA44OUMSMC4ILQ2MGCMXIA34STRUZA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJTGI2TQNBZGQ4KM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJNLQOJPWG33NNVSW45C7N5YGK3S7MNWGSY3L#issuecomment-4132584948> Yes, I've already encountered a problem where warnings and errors aren't being output through services, because the |Service::outputError| procedure is essentially empty, with only |fb_assert(False)| I initially made a commit where service warnings are output to stdout. But now I'm more inclined to redirect all service error output to standard output, something like: void Service::outputError(const char* text) { outputVerbose(text); } @AlexPeshkoff <https://github.com/AlexPeshkoff>, what do you think? Will that be enough?

Yes, for restore - quite enough. Backup is worse - in data mode (.fbk file goes to stdout) that lines will be anyway not shown. Currently I see no simple way to solve an issue just redirecting messages here and there. Something like adding second data pump from services manager to client app of marking data blocks in stdout flow as 'data' and 'warning' should help. But both seem to cause services API incompatibility.

[pavel-zotov]

i have a question about difference between gbak and fbsvcmgr logs.
Comparing them on 6.0.0.1858-c0190d0 (i.e. it includes postfix e1ac987).
Use two commands:

gbak -rep C:\FBTESTING\qa\misc\tmp4test.fbk C:\FBTESTING\qa\misc\tmp4test.tmp ^
   1>tmp-unrestorable_-_gbak-restore-from-FBK.6x1858.log ^
   2>tmp-unrestorable_-_gbak-restore-from-FBK.6x1858.err

== vs ==

C:\FB\60SS\fbsvcmgr localhost:service_mgr action_restore ^
   bkp_file C:\FBTESTING\qa\misc\tmp4test.fbk ^
   dbname C:\FBTESTING\qa\misc\TMP4TEST.TMP ^
   res_replace ^
   1>tmp-unrestorable_-_fbsvcmgr.6x1858.log ^
   2>tmp-unrestorable_-_fbsvcmgr.6x1858.err

Result:

1.1) tmp-unrestorable_-_gbak-restore-from-FBK.6x1858.log -- empty.
1.2) tmp-unrestorable_-_gbak-restore-from-FBK.6x1858.err:

gbak:cannot commit index "PUBLIC"."TEST_X_UNQ"
gbak: ERROR:attempt to store duplicate value (visible to active transactions) in unique index "PUBLIC"."TEST_X_UNQ"
gbak: ERROR:    Problematic key value is (<expression> = 0)
gbak: ERROR:Database is not online due to failure to activate one or more indices.
gbak: ERROR:    Run gfix -online to bring database online without active indices.

2.1) tmp-unrestorable_-_fbsvcmgr.6x1858.log:

gbak:cannot commit index "PUBLIC"."TEST_X_UNQ" 
gbak: ERROR:attempt to store duplicate value (visible to active transactions) in unique index "PUBLIC"."TEST_X_UNQ" 
gbak: ERROR:    Problematic key value is (<expression> = 0) 

2.2) tmp-unrestorable_-_fbsvcmgr.6x1858.err:

Database is not online due to failure to activate one or more indices.
-Run gfix -online to bring database online without active indices.

So, entire output for gbak is redirected to STDERR while for fbsvcmgr it is not so.
Is it expected ?
gh-8793-6x1858_-_-gbak-vs-fbsvcmgr-logs.zip

[XaBbl4]

Is it expected ?

Yes, unfortunately, that's the case now. There's no trivial way to output errors to stderr via services, so they're redirected to standard output (as was the case previously).

[dyemanov]

I'd say it's worth creating a ticket to extend the Services API to separate stdin/stdout/stderr/data streams. Maybe we could do it in v7.