fastlane-community / trainer

Convert xcodebuild plist and xcresult files to JUnit reports

Home Page:https://krausefx.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

can't find results with Xcode 11

bitcoder opened this issue · comments

Hi,
i'm running "fastlane test" in a simple example project and using the "trainer" plugin to generate JUnit XML report.
However, whenever I run fastlane it is not able of finding the results.

This is my "fastlane/Fastfile"

default_platform(:ios)

platform :ios do
  desc "Run tests"
  lane :test do
    scan(scheme: "UnitTest-Calculator",
       output_types: "",
       fail_build: false)

    trainer(output_directory: "build/reports/")
  end

end

This is the output obtained:

+----------------------------+-------------------------------------------------------------------------------------------+
|                                                      Lane Context                                                      |
+----------------------------+-------------------------------------------------------------------------------------------+
| DEFAULT_PLATFORM           | ios                                                                                       |
| PLATFORM_NAME              | ios                                                                                       |
| LANE_NAME                  | ios test                                                                                  |
| SCAN_DERIVED_DATA_PATH     | /Users/smsf/Library/Developer/Xcode/DerivedData/UnitTest-Calculator-fbcsrzqmxnbtunckthqv  |
|                            | azzqktol                                                                                  |
# This file contains the fastlane.tools configuration
| SCAN_GENERATED_PLIST_FILES | []                                                                                        |
| SCAN_GENERATED_PLIST_FILE  |                                                                                           |
+----------------------------+-------------------------------------------------------------------------------------------+
[12:10:06]: No test result files found in directory '.', make sure the file name ends with 'TestSummaries.plist' or '.xcresult'

+------+------------------+-------------+
|           fastlane summary            |
+------+------------------+-------------+
| Step | Action           | Time (in s) |
+------+------------------+-------------+
| 1    | default_platform | 0           |
| 2    | scan             | 23          |
| 💥   | trainer          | 0           |
+------+------------------+-------------+

[12:10:06]: fastlane finished with errors

[!] No test result files found in directory '.', make sure the file name ends with 'TestSummaries.plist' or '.xcresult'

This is the content of the build directory, under the "DerivedData" default one:

$ ls /Users/smsf/Library/Developer/Xcode/DerivedData/UnitTest-Calculator-fbcsrzqmxnbtunckthqvazzqktol/Logs/Test/
LogStoreManifest.plist                                     Run-UnitTest-Calculator-2019.10.24_12-05-37-+0100.xcresult

The "Run-UnitTest-Calculator-2019.10.24_12-05-37-+0100.xcresult" is not a file but a directory instead.
It does not contain any TestSummaries.plist neither a .xcresult file.

Am I missing something basic in the configuration of fastlane or in the build itself to generate .xresult file? As it seems, XCode 11 is generating a .xcresult directory with many files.
From the trainer code, from my understanding it does not go into .xcresult directories under the DerivedData folder.. but anyway, I don't see a specific .xcresult there.

I am facing the same issue. Have you found any work around or any solution?

Unfortunately the only solution was to use another tool: xcpretty.
You can see an example here: https://confluence.xpand-it.com/display/XRAY/Testing+iOS+apps+using+XCTest+in+Swift

We have the same issue with Xcode 11.2 and Mojave 10.14.6.

The current workaround we applied is to copy the last xcresult bundle to the fastlane folder (or project folder to not add path to trainer action…).

trainer parses xcresult bundles. It's ok for that.

Xcode only keep the latest 2 xcresult bundles. If "latest bundle" is not safe enough, for sure, trainer has to check LogStoreManifest.plist.

Unfortunately the only solution was to use another tool: xcpretty.
You can see an example here: https://confluence.xpand-it.com/display/XRAY/Testing+iOS+apps+using+XCTest+in+Swift

Issue I found after switching to rely on xcpretty is it can only handle basic use cases. For me I needed it to work with multiple devices so I put together this plugin

@bitcoder The following configuration worked for me.
The main config is
result_bundle: true
trainer(path: "./fastlane/test_output", output_directory: "./fastlane")

default_platform(:ios)

  platform :ios do
  desc "Run tests"
  lane :test do
    scan(scheme: "UnitTest-Calculator",
      output_types: "",
      fail_build: false,
      result_bundle: true
    )
    trainer(path: "./fastlane/test_output", output_directory: "./fastlane")
  end
end