Add cleanup to K8s resources, and retry for transient errors

[eva-1729]

Closes #391

Description

Adding some error handling and other cleanup, in order to try and catch failures in job creation, as may be happening with Simple Jobs, to stop them from vanishing.

[codecov]

Codecov Report

❌ Patch coverage is 75.00000% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.46%. Comparing base (2bfe5ce) to head (69df03d).

Files with missing lines	Patch %	Lines
job_creator/jobcreator/job_creator.py	73.43%	17 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #415      +/-   ##
==========================================
- Coverage   97.86%   94.46%   -3.40%     
==========================================
  Files           3        7       +4     
  Lines         234      524     +290     
==========================================
+ Hits          229      495     +266     
- Misses          5       29      +24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.