about summary refs log tree commit diff
path: root/pkgs/development/python-modules/pyspark
AgeCommit message (Collapse)AuthorFilesLines
2021-10-19python3Packages.pyspark: update postPatch and add pythonImportsCheckFabian Affolter1-5/+16
2021-10-19python38Packages.pyspark: 3.1.2 -> 3.2.0R. RyanTM1-2/+2
2021-06-22python3Packages.pyspark: 3.1.1 -> 3.1.2Martin Weinelt1-2/+2
2021-04-03python3Packages.pyspark: 3.0.2 -> 3.1.1Frederik Rietdijk1-2/+2
2021-02-26python3Packages.pyspark: 3.0.1 -> 3.0.2Frederik Rietdijk1-2/+2
2021-01-25treewide: remove stdenv where not neededPavol Rusnak1-1/+1
2021-01-11treewide: with stdenv.lib; in meta -> with lib;Profpatsch1-2/+2
Part of: https://github.com/NixOS/nixpkgs/issues/108938 meta = with stdenv.lib; is a widely used pattern. We want to slowly remove the `stdenv.lib` indirection and encourage people to use `lib` directly. Thus let’s start with the meta field. This used a rewriting script to mostly automatically replace all occurances of this pattern, and add the `lib` argument to the package header if it doesn’t exist yet. The script in its current form is available at https://cs.tvl.fyi/depot@2f807d7f141068d2d60676a89213eaa5353ca6e0/-/blob/users/Profpatsch/nixpkgs-rewriter/default.nix
2020-11-23python3Packages.pyspark: Fix out of date postPatchJoshua Skrzypek1-2/+1
Pyspark switched to pinning py4j==0.10.9 with v3.0.0 - see this commit: https://github.com/apache/spark/\ commit/fc4e56a54c15e20baf085e6061d3d83f5ce1185d This meant that since the bump to pyspark v3.0.0 - in this commit: https://github.com/NixOS/nixpkgs/\ commit/5181547ae6624b462919a806c4d0888e6e4630f4 - the patch was no longer matching on the 'py4j==0.10.7' string that was working previously. The failing patch went unnoticed previously because the version of py4j pinned by pyspark>=3.0.0 was the same as the py4j provided by nixpkgs. However, a recent PR (#101636) bumped the version of py4j to 0.10.9.1 in this commit: https://github.com/NixOS/nixpkgs/\ commit/43a91282d66223c5cb978d53fbe1033f56dd7f2b which caused the version pinned by pyspark to no longer match the version provided by nixpkgs. FWIW, @jonringer flagged this issue on another PR that tried to bump py4j: #100623. My solution here was to upgrade the patch's target string to match the version found in pyspark's current setup.py.
2020-10-17python37Packages.pyspark: 3.0.0 -> 3.0.1R. RyanTM1-2/+2
2020-08-16python: pyspark: 2.4.6 -> 3.0.0Frederik Rietdijk1-2/+2
2020-06-11python: pyspark: 2.4.5 -> 2.4.6Frederik Rietdijk1-2/+2
2020-05-11python3Packages.pyspark: 2.4.4 -> 2.4.5Frederik Rietdijk1-2/+2
2020-04-10treewide: Per RFC45, remove all unquoted URLsMichael Reilly1-1/+1
2019-10-18python: pyspark: 2.4.3 -> 2.4.4Frederik Rietdijk1-2/+2
2019-06-15python: pyspark: 2.4.2 -> 2.4.3Frederik Rietdijk1-2/+2
2019-04-24python: pyspark: 2.4.1 -> 2.4.2Frederik Rietdijk1-2/+2
2019-04-18python: pyspark: 2.4.0 -> 2.4.1Frederik Rietdijk1-2/+2
2019-01-30pyspark: 2.3.2 -> 2.4.0, fix version boundsMatthew Bauer1-2/+5
py4j seems to work fine with this version
2018-10-13python: pyspark: 2.3.1 -> 2.3.2Frederik Rietdijk1-2/+2
2018-06-22python: pyspark: 2.3.0 -> 2.3.1Frederik Rietdijk1-2/+2
2018-03-17pythonPackages.pyspark: Init at 2.3.0Shea Levy1-0/+28