kernel-cve-check: Improve error handling when fetch fails

[teradat]

Purpose of pull request

If fetching cip-kernel-sec repository fails, the CVE checks cannot be performed. In this case, output backtrace [1] because there is insufficient error checking.
So improve to error handling.

[1]

ERROR: cve-update-nvd2-native-1.0-r0 do_populate_cve_db: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:update_cip_kernel_sec(d)
     0003:
File: '<path-to>/meta-emlinux/classes/kernel-cve-check.bbclass', lineno: 75, function: update_cip_kernel_sec
     0071:        os.mkdir(kernel_cve_check_dir)
     0072:
     0073:    if not os.path.isdir(cip_kernel_sec_path):
     0074:        # first run
 *** 0075:        runfetchcmd("git clone %s cip-kernel-sec" % git_uri, d,  workdir=kernel_cve_check_dir)
     0076:        remove_remote(os.path.join(cip_kernel_sec_path, "conf"))
     0077:        runfetchcmd("git update-index --skip-worktree conf/remotes.yml", d, workdir=cip_kernel_sec_path)
     0078:    else:
     0079:        runfetchcmd("git pull", d, workdir=cip_kernel_sec_path)
File: '<path-to>/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 886, function: runfetchcmd
     0882:                bb.utils.remove(f, True)
     0883:            except OSError:
     0884:                pass
     0885:
 *** 0886:        raise FetchError(error_message)
     0887:
     0888:    return output
     0889:
     0890:def check_network_access(d, info, url):
Exception: bb.fetch2.FetchError: Fetcher failure: Fetch command <...snip...>
Cloning into 'cip-kernel-sec'...
fatal: unable to access 'https://gitlab.com/cip-project/cip-kernel/cip-kernel-sec.git/': Failed to connect to <proxy-server> port <proxy-port>: Connection refused

ERROR: cve-update-nvd2-native-1.0-r0 do_populate_cve_db: Fetcher failure: Fetch command <...snip...>
Cloning into 'cip-kernel-sec'...
fatal: unable to access 'https://gitlab.com/cip-project/cip-kernel/cip-kernel-sec.git/': Failed to connect to <proxy-server> port <proxy-port>: Connection refused

ERROR: cve-update-nvd2-native-1.0-r0 do_populate_cve_db: Function failed: update_cip_kernel_sec
ERROR: Logfile of failure stored in: <path-to>/build/tmp-glibc/work/x86_64-linux/cve-update-nvd2-native/1.0-r0/temp/log.do_populate_cve_db.2578823
ERROR: Task (<path-to>/meta-emlinux/recipes-core/cve-update/cve-update-nvd2-native.bb:do_populate_cve_db) failed with exit code '1'

Background

When cve-check cannot be performed, there are two ways of thinking depending on the purpose of bitbake:

1. The purpose is build, want to continue to build
   e.g. bitbake core-image-minimal
2. The purpose is cve-check, want to immediately terminate with an error
   e.g. bitbake linux-base -c cve_check

Add "CVE_CHECK_ERROR_ON_FAILURE" variable to satisfy these wants.

• Set "0" (is default): skip the CVE check and continue with build of bitbake
  By disabling "CVE_CHECK_DB_FILE" variable, CVE check will be skipped in Poky's do_cve_check() function.
  This is the same behavior as if the NVD database download failed in Poky, skip the CVE check and continue with build.
• Set "1": bitbake return fatal error immediately
  Immediately exit with bb.fatal().

Details of improvements

In summary, the following changes in this commit:

• Add exception handling to update_cip_kernel_sec()
• Add check of repository synced
  (Even if fetching cip-kernel-sec repository fails,) A successfully fetched local repository may exist, so if the synced timestamp is today, it is considered a valid repository.
• Add error handling logic for "CVE_CHECK_ERROR_ON_FAILURE" variable

Test

How to test

1. local.conf setting
   Add the following to local.conf.

   MACHINE = "qemuarm64"
   INHERIT += " cve-check kernel-cve-check"
   

   And, for purpose of test, modify CVE_CHECK_ERROR_ON_FAILURE.

   • For skip behavior

     CVE_CHECK_ERROR_ON_FAILURE = "0"
     

   • For fatal behavior

     CVE_CHECK_ERROR_ON_FAILURE = "1"
     

2. In preparation, complete the build of linux-base

   $ bitbake linux-base; echo "bitbake return-code: $?"
   

3. Preparing for testing

   • Remove the local repository of cip-kernel-sec

     $ rm -fr ../downloads/CVE_CHECK/KERNEL/cip-kernel-sec
     

   • Make cip-kernel-sec fetch fails

     $ touch ../downloads/CVE_CHECK/KERNEL/cip-kernel-sec
     

4. Build linux-base and check log

   bitbake -v linux-base; echo "bitbake return-code: $?"
   

Test result

For skip behavior

Displayed skipping CVE check message and bitbake succeeds.

build$ bitbake -v linux-base; echo "bitbake return-code: $?"
...snip...
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: cve-update-nvd2-native-1.0-r0 do_populate_cve_db: CVE database recently updated, skipping
NOTE: linux-base-4.19-r0 do_cve_check: kernel_cve_check: cip-kernel-sec repository sync failure, skipping CVE check
NOTE: linux-base-4.19-r0 do_cve_check: No CVE database found, skipping CVE check
NOTE: Tasks Summary: Attempted 694 tasks of which 691 didn't need to be rerun and all succeeded.
bitbake return-code: 0

For fatal behavior

By bb.fatal(), displayed ERROR message and bitbake stops.

build$ bitbake -v linux-base; echo "bitbake return-code: $?"
...snip...
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: cve-update-nvd2-native-1.0-r0 do_populate_cve_db: CVE database recently updated, skipping
ERROR: linux-base-4.19-r0 do_cve_check: kernel_cve_check: cip-kernel-sec repository sync failure
ERROR: linux-base-4.19-r0 do_cve_check: 
ERROR: linux-base-4.19-r0 do_cve_check: Function failed: kernel_cve_check
ERROR: Logfile of failure stored in: /project/security-update/test1/build/tmp-glibc/work/qemuarm64-emlinux-linux/linux-base/4.19-r0/temp/log.do_cve_check.154098
ERROR: Task (/project/security-update/test1/build/../repos/meta-debian/recipes-kernel/linux/linux-base_git.bb:do_cve_check) failed with exit code '1'
NOTE: Tasks Summary: Attempted 693 tasks of which 691 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /project/security-update/test1/build/../repos/meta-debian/recipes-kernel/linux/linux-base_git.bb:do_cve_check
Summary: There were 3 ERROR messages shown, returning a non-zero exit code.
bitbake return-code: 1