26 Commits

Author SHA1 Message Date
Mark J. Hoy
51a2b9cc19
Add 9.1.0 Snapshot to Build and Fix test_ml_model Tests to Normalized Expected Scores if Min Score is Less Than Zero (#777)
* normalized expected scores if min is < 0

* only normalize scores for ES after 8.19+ / 9.1+

* add 9.1.0 snapshot to build matrix

* get min score from booster trees

* removing typing on function definition

* properly flatten our tree leaf scores

* simplify getting min score

* debugging messages

* get all the matches in better way

* Fix model score normalization.

* lint

* lint again

* lint; correct return for bounds map/list

* revert to Aurelian's fix

* re-lint :/

---------

Co-authored-by: Aurelien FOUCRET <aurelien.foucret@elastic.co>
2025-04-23 15:53:32 +00:00
Quentin Pradet
87380ef716
Release 9.0.0
Co-authored-by: Miguel Grinberg <miguel.grinberg@gmail.com>
2025-04-16 15:21:04 +04:00
Quentin Pradet
acdeeeded2
Allow nox 2025.02.09 (#754) 2025-02-12 16:33:59 +04:00
Quentin Pradet
8350f06ea8
Fix pipeline labels (#751) 2025-02-12 15:07:51 +04:00
Quentin Pradet
871e52b37a
Pin nox to avoid session.env issue (#753) 2025-02-11 18:36:57 +04:00
Bart Broere
75c57b0775
Support Pandas 2 (#742)
* Fix test setup to match pandas 2.0 demands

* Use the now deprecated _append method

(Better solution might exist)

* Deal with numeric_only being removed in metrics test

* Skip mad metric for other pandas versions

* Account for differences between pandas versions in describe methods

* Run black

* Check Pandas version first

* Mirror behaviour of installed Pandas version when running value_counts

* Allow passing arguments to the individual asserters

* Fix for method _construct_axes_from_arguments no longer existing

* Skip mad metric if it does not exist

* Account for pandas 2.0 timestamp default behaviour

* Deal with empty vs other inferred data types

* Account for default datetime precision change

* Run Black

* Solution for differences in inferred_type only

* Fix csv and json issues

* Skip two doctests

* Passing a set as indexer is no longer allowed

* Don't validate output where it differs between Pandas versions in the environment

* Update test matrix and packaging metadata

* Update version of Python in the docs

* Update Python version in demo notebook

* Match noxfile

* Symmetry

* Fix trailing comma in JSON

* Revert some changes in setup.py to fix building the documentation

* Revert "Revert some changes in setup.py to fix building the documentation"

This reverts commit ea9879753129d8d8390b3cbbce57155a8b4fb346.

* Use PANDAS_VERSION from eland.common

* Still skip the doctest, but make the output pandas 2 instead of 1

* Still skip doctest, but switch to pandas 2 output

* Prepare for pandas 3

* Reference the right column

* Ignore output in tests but switch to pandas 2 output

* Add line comment about NBVAL_IGNORE_OUTPUT

* Restore missing line and add stderr cell

* Use non-private method instead

* Fix indentation and parameter issues

* If index is not specified, and pandas 1 is present, set it to True

From pandas 2 and upwards, index is set to None by default

* Run black

* Newer version of black might have different opinions?

* Add line comment

* Remove unused import

* Add reason for ignore statement

* Add reason for skip

---------

Co-authored-by: Quentin Pradet <quentin.pradet@elastic.co>
2025-02-04 17:43:43 +04:00
Bart Broere
9b5badb941
Drop Python 3.8 support and introduce Python 3.12 CI/CD (#743) 2025-01-22 21:55:57 +04:00
Quentin Pradet
f99adce23f
Build documentation using Docker again (#746) 2025-01-14 18:16:39 +04:00
Quentin Pradet
d1e533ffb9
Fix Docker image build on Linux (#728)
* Fix Docker image build on Linux

* Build Docker images in CI

* Fix bash syntax

* Only load, not push

* Parallelize docker build

It's currently the slowest step.

* Only build Linux images
2024-10-02 10:33:35 +04:00
David Kyle
ec66b5f320
Add ES 8.16 and 8.15.2 to test matrix (#725) 2024-09-27 13:37:31 +01:00
Quentin Pradet
64d05e4c68
Restore public Dockerfile (#722) 2024-09-25 12:49:46 +04:00
Aurélien FOUCRET
bee6d0e1f7
Remove input fields from exported LTR models (#708) 2024-07-05 14:31:22 +02:00
David Kyle
c16e36c051
Add Python 3.11 to support matrix (#681) 2024-03-27 10:34:35 +00:00
David Kyle
ae0bba34c6
Upgrade torch to 2.1.2 (#671)
Compatible with Elasticsearch 8.13 where the same upgrade has been made
2024-03-26 10:06:50 +00:00
Quentin Pradet
fd2ceab846
Run Buildkite docs jobs in pull requests from forks (#652) 2024-01-31 20:55:19 +04:00
Aurélien FOUCRET
0f91224daf
Add 8.12 to CI and remove 8.10 (#647) 2024-01-04 10:06:19 -05:00
David Kyle
add61a69ec
Update CI machine types to N2 (#634)
Use `n2-standard-2` for lint and doc builds
Use `n2-standard-4` for tests
2023-11-21 11:33:04 +00:00
Valeriy Khakhutskyy
6cecb454e3
[ML] Better memory estimation for NLP models (#568)
This PR adds an ability to estimate per deployment and per allocation memory usage of NLP transformer models. It uses torch.profiler and performs logs the peak memory usage during the inference.

This information is then used in Elasticsearch to provision models with sufficient memory (elastic/elasticsearch#98874).
2023-11-06 12:18:20 +01:00
Quentin Pradet
0c0a8ab19f
Bump tested stack versions (#621) 2023-10-11 19:48:47 +02:00
Quentin Pradet
9273636026
Reduce Docker image size and support arm64 (#615)
Co-authored-by: David Olaru <dolaru@elastic.co>

* Reduce Docker image size from 4.8GB to 2.2GB

* Use torch+cpu variant if target platform is linux/amd64

Avoids downloading large & unnecessary NVIDIA deps defined in the package on PyPI

* Build linux/arm64 image using buildx and QEMU
2023-10-05 18:43:52 +04:00
Quentin Pradet
352e31ed14
Add Buildkite pipeline to push Docker image (#613)
* Add Buildkite pipeline to push Docker image

* Fix lint

* Fix Read the Docs build

* Replace distutils with packaging
2023-10-03 14:39:54 +02:00
Quentin Pradet
eb69496627
Add dummy pipeline to prepare publishing a Docker image (#590) 2023-09-06 07:12:06 +02:00
Quentin Pradet
4d2c6e2f4d
Fix Buildkite builds on pull requests (#589) 2023-09-05 12:20:24 +04:00
Enrico Zimuel
08b7fac32b
Updated test to ES 8.9-SNAPSHOT 2023-08-23 13:53:15 +02:00
Fernando Briano
7ad1f430e4
[CI] Adds buildkite pull requests configuration (#570) 2023-07-26 13:43:40 +01:00
Fernando Briano
2134c71ab4
Add Buildkite configuration (#515)
* [CI] Adds Buildkite configuration
* Removes GitHub Actions
* Moves lint and docs tasks to Buildkite
2023-07-17 14:08:41 +01:00