[Dart] Add UInt8List support, allowing for type:string format:binary definitions #13451

0xNF · 2022-09-17T09:55:01Z

Summary

tl;dr UInt8List support, which Dart-Dio has, has been added to the regular Dart2 generator.

Details

Changes to the Generators:

To support UInt8List in the generator, Typed Data imports had to be added. Additionally, some OpenAPI data types have been switched to Uint8List.

[AbstractDartCodeGen.java]: added Uint8List/dart:typed_data to the imports.
[AbstractDartCodeGen.java]: Changed typeMapping of {file, binary} to Uint8List away from MultiPartFile
[AbstractDartCodeGen.java]: Copied Multipart/Uint8list differentiation in postProcessOperationsWithModels from DartDioClientCodegen.java
[dart2/apilib.mustache]: Added import 'dart:typed_data';

Changes to the API classes

To support Uint8List in the generated API output, parts of the *api.dart files had to be refactored. Presently Dart2 relies on passing utf8 decoded response byte strings through its API calls, but this leads to errors because not all response objects are json strings. To handle this, I refactored many of the calls to take Uint8List's straight from response.bodyBytes. Decoding into a String object doesn't happen until it needs to, which is right before passing the object down into the Json Decoding section.

[api_client.dart]: deserialize and deserializeAsync now take Uint8List as their default input
[api_helper.dart]: removed _decodeBodyBytes in favor of just using response.bodyBytes
[api_helper.dart]: added decodeError method to deal with the byte response, which decodes the resposne into a utf-8 String.

To confirm this change, use the following specfile:

specfile

openapi: 3.0.3
info:
  version: "1.1"
  title: Dart Uint8list Demo
servers:
  - url: "localhost"
    variables:
      host:
        default: localhost
paths:
  /getData:
    get:
      operationId: GetData
      description: "Should return a Uint8List of bytes"
      responses:
        "200":
          description: Binary data
          content:
            application/octet-stream:
              schema:
                type: string
                format: binary
    put:
      operationId: PutData
      description: "Should push a Multipart File"
      requestBody:
        content:
          multipart/form-data:
            schema:
              type: object
              properties:
                fileContent:
                  type: string
                  format: binary
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                type: string
  /item:
    get:
      operationId: GetItem
      description: "Should return an Item"
      responses:
        "200":
          description: items
          content:
            application/json:
              schema:
                $ref: "#components/schemas/item"
components:
  schemas:
    Item:
      type: object
      description: "Some json thing"
      required:
        - intField
        - doubleField
        - stringField
        - dateField
        - timeField
        - dateTimeField
        - boolField
        - b64Field
        - intArrayField
      properties:
        intField:
          type: integer
          format: int32
        doubleField:
          type: number
          format: float
        stringField:
          type: string
        dateField:
          type: string
          format: date
        timeField:
          type: string
          format: time
        dateTimeField:
          type: string
          format: date-time
        boolField:
          type: boolean
        b64Field:
          type: string
          format: byte
        intArrayField:
          type: array
          items:
            type: integer
            format: int32

PR checklist

Read the contribution guidelines.
Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
Run the following to build the project and update samples:
File the PR against the correct branch: master (6.1.0) (minor release - breaking changes with fallbacks), 7.0.x (breaking changes without fallbacks)
If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

@jaumard @josh-burton @amondnet @sbu-WBT @kuhnroyal @agilob @ahmednfwela

ahmednfwela

there are alot of whitespace changes here that are polluting the PR, it would be great if the PR had the bare minimum changes

ahmednfwela · 2022-09-17T13:40:37Z

samples/openapi3/client/petstore/dart2/petstore_client_lib/lib/api/pet_api.dart

 class PetApi {
  PetApi([ApiClient? apiClient]) : apiClient = apiClient ?? defaultApiClient;

  final ApiClient apiClient;

  /// Add a new pet to the store
  ///
-  /// 
+  ///


please avoid whitespace changes since they pollute the PR

ahmednfwela · 2022-09-17T13:41:32Z

samples/openapi3/client/petstore/dart2/petstore_client_lib/lib/api/pet_api.dart

  ///
  /// Note: This method returns the HTTP [Response].
  ///
  /// Parameters:
  ///
  /// * [Pet] pet (required):
  ///   Pet object that needs to be added to the store
-  Future<Response> addPetWithHttpInfo(Pet pet,) async {
+  Future<Response> addPetWithHttpInfo(
+    Pet pet,


whitespace changes here too

The newest version of this branch should have corrected the extraneous whitespace.

0xNF · 2022-09-17T13:51:17Z

When I get home I'll try to fix it, but I didn't touch any of those files by hand. I only ran the generate samples script. I didn't even open these in my editor.

ahmednfwela · 2022-09-18T10:04:58Z

modules/openapi-generator/src/main/resources/dart2/api_client.mustache

+      return responseBytes;
+    }
+
+    final decodedString = const Utf8Decoder().convert(responseBytes);
    // If the expected target type is String, nothing to do...
    return targetType == 'String'
      ? json


shouldn't this return decodedString?

Eagle-eye'd catch there. json isn't even a variable in the new method anymore, my mistake New push of the branch has it returning decodedString now.

* Adds Uint8List as a valid return type for type: string format: binary definitions * Adds support for using bodyBytes instead of body for deserializing response data

ahmednfwela · 2022-09-19T07:39:47Z

@kuhnroyal @wing328 Can you check why the tests were passing here when there was a compile error for the generated code?

ahmednfwela · 2022-09-19T07:45:45Z

@0xNF did you make any changes in the java files? I can't see them in the PR

0xNF · 2022-09-19T08:11:11Z

The Java changes seem to have gotten lost. Sorry this PR is kinda messy. I'm flipping between too many branches rn.

wing328 · 2022-09-19T08:31:23Z

@kuhnroyal @wing328 Can you check why the tests were passing here when there was a compile error for the generated code?

The test did run and passed. Does it has anything to do with Dart version?

      - uses: dart-lang/setup-dart@v1
        with:
          sdk: 2.14.0

That's what we use in the CI.

ahmednfwela · 2022-09-19T08:47:15Z

dart 2 uses 2.12 in the generated code, which 2.14 should be able to run

wing328 · 2022-09-19T08:48:36Z

Can you please paste the compilation error here as well? I'll try to repeat that locally.

0xNF · 2022-09-19T09:51:25Z

It's probably because I force pushed the update to my own branch, and the tests didn't re-run here.

wing328 · 2022-09-19T11:58:18Z

It's probably because I force pushed the update to my own branch, and the tests didn't re-run here.

To trigger the run, please update the samples and the Github action (dart job) will run again.

0xNF · 2022-09-20T01:24:28Z

Should I be regenerating all samples? Up til now I've just been regenerating the relevant samples for my changes (Dart). But then the CI complains.

wing328 · 2022-09-20T13:28:38Z

yes please update the samples as the CI fails: https://github.com/OpenAPITools/openapi-generator/actions/runs/3086648029/jobs/4991237644

zmoshansky · 2023-01-10T23:34:58Z

@0xNF @wing328 Is there any movement on this? It looks like this PR is functionally ready to land. Just needing some clean-up.

Currently, I'm facing an issue around this as passing a UInt8List to a generated API endpoint causes the following error:

DioError [DioErrorType.other]: Bad state: No serializer for 'Uint8List'.

Relevant Code (Generated API's model.dart):

    yield serializers.serialize(
      object.appBinaryData,
      specifiedType: const FullType(Uint8List),
    );

ahmednfwela · 2023-01-10T23:49:47Z

@zmoshansky this PR is for dart generator, and it seems like you are using the dart-dio generator, please create a new Issue with reproducible sample

0xNF changed the title ~~Dart: Add UInt8List support, allowing for type:string format:binary definitions~~ [Dart] Add UInt8List support, allowing for type:string format:binary definitions Sep 17, 2022

ahmednfwela suggested changes Sep 17, 2022

View reviewed changes

0xNF force-pushed the fix_dart_uint8list branch from 5e715aa to 819c9f1 Compare September 18, 2022 04:40

0xNF requested a review from ahmednfwela September 18, 2022 04:44

ahmednfwela suggested changes Sep 18, 2022

View reviewed changes

0xNF added 2 commits September 19, 2022 11:17

[Dart] Add Uint8List support to Dart generator (OpenAPITools#12161)

f33ff15

* Adds Uint8List as a valid return type for type: string format: binary definitions * Adds support for using bodyBytes instead of body for deserializing response data

[Dart] Update samples

d5ed19f

0xNF force-pushed the fix_dart_uint8list branch from 819c9f1 to d5ed19f Compare September 19, 2022 02:19

wing328 added Client: Dart Enhancement: General labels Sep 19, 2022

wing328 added this to the 6.1.1 milestone Sep 19, 2022

[Dart] Re-added in missing Uint8 list imports in DartCodegen

235d4fd

wing328 modified the milestones: 6.1.1, 6.2.1 Sep 24, 2022

wing328 modified the milestones: 6.2.1, 6.3.0 Nov 1, 2022

zmoshansky mentioned this pull request Jan 12, 2023

[BUG] [DART-DIO] No serializer for 'Uint8List' #14446

Open

6 tasks

wing328 modified the milestones: 6.3.0, 6.3.1 Jan 20, 2023

wing328 modified the milestones: 6.4.0, 6.5.0 Feb 19, 2023

wing328 modified the milestones: 6.5.0, 6.6.0 Apr 1, 2023

wing328 modified the milestones: 6.6.0, 7.0.0 May 11, 2023

wing328 modified the milestones: 7.0.0, 7.0.1 Aug 25, 2023

wing328 modified the milestones: 7.0.1, 7.1.0 Sep 20, 2023

wing328 modified the milestones: 7.1.0, 7.2.0 Nov 13, 2023

wing328 modified the milestones: 7.2.0, 7.3.0 Dec 22, 2023

wing328 modified the milestones: 7.3.0, 7.4.0 Feb 8, 2024

wing328 modified the milestones: 7.4.0, 7.5.0 Mar 11, 2024

wing328 modified the milestones: 7.5.0, 7.6.0 Apr 17, 2024

Uh oh!

[Dart] Add UInt8List support, allowing for type:string format:binary definitions #13451

Are you sure you want to change the base?

[Dart] Add UInt8List support, allowing for type:string format:binary definitions #13451

Uh oh!

Conversation

0xNF commented Sep 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Details

Changes to the Generators:

Changes to the API classes

PR checklist

Uh oh!

ahmednfwela left a comment

Choose a reason for hiding this comment

Uh oh!

ahmednfwela Sep 17, 2022

Choose a reason for hiding this comment

Uh oh!

ahmednfwela Sep 17, 2022

Choose a reason for hiding this comment

Uh oh!

0xNF Sep 18, 2022

Choose a reason for hiding this comment

Uh oh!

0xNF commented Sep 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ahmednfwela Sep 18, 2022

Choose a reason for hiding this comment

Uh oh!

0xNF Sep 19, 2022

Choose a reason for hiding this comment

Uh oh!

ahmednfwela commented Sep 19, 2022

Uh oh!

ahmednfwela commented Sep 19, 2022

Uh oh!

0xNF commented Sep 19, 2022

Uh oh!

wing328 commented Sep 19, 2022

Uh oh!

ahmednfwela commented Sep 19, 2022

Uh oh!

wing328 commented Sep 19, 2022

Uh oh!

0xNF commented Sep 19, 2022

Uh oh!

wing328 commented Sep 19, 2022

Uh oh!

0xNF commented Sep 20, 2022

Uh oh!

wing328 commented Sep 20, 2022

Uh oh!

zmoshansky commented Jan 10, 2023

Uh oh!

ahmednfwela commented Jan 10, 2023

Uh oh!

Uh oh!

0xNF commented Sep 17, 2022 •

edited

Loading

0xNF commented Sep 17, 2022 •

edited

Loading