[Snyk] Security upgrade nltk from 3.3 to 3.9.3

[brookecastleberry]

Snyk has created this PR to fix 1 vulnerabilities in the pip dependencies of this project.

Snyk changed the following file(s):

• pip-sample/requirements.txt

⚠️ Warning

jupyter-console 6.6.3 has requirement ipykernel>=6.14, but you have ipykernel 5.0.0.
jupyter-console 6.6.3 has requirement jupyter-core!=5.0.*,>=4.12, but you have jupyter-core 4.4.0.
jupyter-console 6.6.3 has requirement traitlets>=5.4, but you have traitlets 4.3.2.
jupyter-console 6.6.3 has requirement jupyter-client>=7.0.0, but you have jupyter-client 5.2.3.
Jinja2 2.10 requires MarkupSafe, which is not installed.
ipython 7.34.0 has requirement jedi>=0.16, but you have jedi 0.12.1.

Breaking Change Risk

Notice: This assessment is enhanced by AI.

---

Important

• Check the changes in this PR to ensure they won't cause issues with your project.
• Max score is 1000. Note that the real score may have changed since the PR was raised.
• This PR was automatically created by Snyk using the credentials of a real user.
• Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

---

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

---

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Eval Injection

[brookecastleberry]

This upgrade from NLTK 3.3 to 3.9.3 spans several minor versions and includes significant breaking changes, behavioral modifications, and changes to Python version support.

High Risk - Breaking Changes:

• Model Data Format Change: Starting with version 3.9, several pre-trained models (including the popular punkt tokenizer) no longer use pickle files to address a security vulnerability (CVE-2024-39705). This is a breaking change. You will need to download the new model data. For example, the punkt model is now punkt_tab. [3, 7]

Medium Risk - Behavioral Changes & Environment:

• Python Version Support: Support for older Python versions has been dropped. NLTK 3.9.2+ requires Python 3.9 or newer and drops support for Python 3.8 and older. [1, 9] Your environment must be using a compatible Python version.
• WordNet Sorting: As of version 3.9, WordNet functions no longer automatically sort synsets and relations. [1, 3] If your code relies on a sorted output, you must now sort the results explicitly in your own functions.
• METEOR Evaluation: Since version 3.6.5, the METEOR evaluation metric requires pre-tokenized input. [1] Passing raw strings to this function may lead to incorrect results or errors.

Recommendation:

1. Ensure your environment is running a supported Python version (3.9+).
2. After upgrading, be prepared to re-download NLTK data packages (e.g., nltk.download('popular') or nltk.download('punkt')) to get the new pickle-free model formats.
3. Review any code that uses WordNet to check if it relies on the previously sorted output.
4. Verify that any usage of the METEOR metric is passing tokenized input.

Notice 🤖: This content was augmented using artificial intelligence. AI-generated content may contain errors and should be reviewed for accuracy before use.