You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
DJL is a new Java framework that build on top of any engines. It brings the ease for Java developers to have close to numpy experience in Java. It introduced an interface that defined Java to train and run inference on different ML/DL models.
In the engine layer, we implemented the MXNet specific engine that allows users to achieve most of the up-to-date functionalities:
MXNet specific
deep-numpy ops (MXNet): numpy operators introduced in MXNet 1.6
autograd (MXNet) : imperative training are supported by doing autogradient collections
Block concept (MXNet Gluon): Java blocks for training and inference
CachedOp (MXNet): new Symbolic Inference Engine
DJL
Full training support: We support on Imperative/Symbolic training in Java.
Memory collection: NDManager introduced to lively track and collect memory, ensured 100 hrs stable run in prod.
MKLDNN: All ops and NDArray are built on top of MKLDNN acceleration
ModelZoo: Model zoo is a central hub for all model storage, it contains model files, preprocessing and post processing logics.
Maintainance
Better loading experience: All native MXNet files are uploaded to maven and users can easily switch different engine versions.
JNA simplification: JNI layers are generated to save maintainance time and brought more logic into Java layer.
With the benefit listed above, I would recommend Option 3 to go for the DJL path since it already covered most up-to-date MXNet feature and supporting all different symbolic/imperative training/inference.
For Option 4: I am also thinking of bring our JNA layer back to MXNet so the community can build up their own Java/Scala frontend if they don't like DJL.
I propose option 1 and 2 since it took us a lot of efforts to bring MXNet to Scala originally and there are already adopters of Scala API in industries (some may not have been disclosed yet). But I am open to other options. Not familiar with DJL though but I assume @frankfliu and @lanking520 are the masters behind DJL.
@lanking520 thanks for the clarification above. A further question - How do you envision a current Scala MXNet user migrate their code? Is it going to be mostly reusable or is it going to be a complete rewrite for them?
It is going to be closer to a complete rewrite. On the other hand, making a new Scala API would be imperative instead of symbolic and I think there are going to be a lot of operator changes to better match numpy in 2.0. I don't think the migration costs for a Scala 2.0 would be that much less anyway
For users who don't want a full rewrite, they can continue using an old release or whatever new releases we make on the v1.x branch.
For the Clojure package. It is a lot easier to interop with Java than with Scala - so if the the base is Java that everything is using - it will be better for Clojure.
+1 for option 1 and 2. Also +1 for 4 as long as it doesn't add a dependency
My concerns on 3 and 4 are that DJL is a separate project which has its own release cycle. Having it to support MXNet's inference will cause delays as DJL upgrades to the latest version. This will also complicate the testing and validation.
Overall, I think a minimum set of API for at least inference is needed for MXNet JVM ecosystem users.
Another data point is that all of our Scala tests fail randomly with src/c_api/c_api_profile.cc:141: Check failed: !thread_profiling_data.calls_.empty():, so there seem to be some underlying issues.
Another data point is that we currently only support OpenJDK 8 but the JVM languages are broken with OpenJDK 11 which is used on Ubuntu 18.04 for example. See #18153
BTW, there's one thing we've neglected to cover. I was under the impression that MXNet was using Cython to access the C API for its Python binding, but it looks like it's using ctypes.
I don't really want to deal with CI, especially Jenkins, it's a major time sink and completely unnecessary with services like GitHub Actions these days
It's also fine to use Github Actions if that's easier for you. The main reason for using Jenkins is that the MXNet test suite is too large for a free service such as Github Actions and that there are also GPU tests involved. Java tests can initially run on Github Actions and be migrated later to Jenkins based on need.
I've updated my fork with a workflow for Java build on GitHub Actions: saudet@2be0540
Please let me know what you think of this!
It's currently building and testing for Linux (CentOS 7), Mac, and Windows on x86_64 with and without CUDA: https://github.com/saudet/incubator-mxnet/actions/runs/437312011
(It looks like the build doesn't work for CUDA 11.1 with Visual Studio 2019 yet, but that's unrelated to Java.)
In any case, the javadoc secondary artifact also gets deployed as part of the build there.
Where does the publishing to the main site happen? Somewhere in here by the looks of it: https://github.com/apache/incubator-mxnet/tree/master/ci/publish/website
We can fetch the latest javadoc archive this way, so I assume we could add that to the scripts?
It's also fine to use Github Actions if that's easier for you. The main reason for using Jenkins is that the MXNet test suite is too large for a free service such as Github Actions and that there are also GPU tests involved. Java tests can initially run on Github Actions and be migrated later to Jenkins based on need.
For that, GitHub Actions now support self-hosted runners, where we just need to provision some machines on the cloud somewhere, and install the equivalent of Jenkins Agent on them, and that's it. Much easier than maintaining Jenkins.
Thank you @saudet. You can take a look at https://infra.apache.org/publishing-maven-artifacts.html for more information on the Apache Software Foundation (ASF) maven artifact publishing process. Summary: Release candidate artifacts are pushed to a staging area and can be promoted after the release vote passed.
For that, GitHub Actions now support self-hosted runners, where we just need to provision some machines on the cloud somewhere, and install the equivalent of Jenkins Agent on them, and that's it. Much easier than maintaining Jenkins.
Github Actions isn't very mature yet. You can see in the doc that "Self-hosted runners on GitHub do not have guarantees around running in ephemeral clean virtual machines, and can be persistently compromised by untrusted code in a workflow." I don't think that's acceptable for projects accepting contributions from the general public.
As libgfortran has changed their ABI a few times over the years, you will need to include libgfortran.so in the jar (which we can distribute under AL2 License thanks to the GCC Runtime Library Exception). However, you must not include libquadmath.so (dependency of libgfortran.so) as it is GPL licensed.
For the gpu version mxnet-2.0-20201222.141246-19-linux-x86_64-gpu.jar, would it make sense to use cu110 instead of gpu if built with cuda 11.0 etc?
Yes I think they are mentioning the same security problem we are having with our jenkins slaves. Any user could run arbitrary code and install a rootkit. Hence the separation towards restricted slaves.
So from that point of view, I don't consider the github actions self runner any less secure than our jenkins slaves. But of course still insecure.
The problem with runners I had in mind is that there used to be no API to start new instances for each job, but rather that the instances had to be up and running all the time and would be re-used for all jobs. Thus any compromise would be truly persistent. We don't do that in our Jenkins setup, where instances are terminated time-to-time.
But I just checked the Github documentation and Microsoft team has resolved this issue and now provides an API that can provision new runners upon demand. So if there are volunteers, it should be fine to migrate to Github Actions. For example, https://040code.github.io/2020/05/25/scaling-selfhosted-action-runners
Thank you @saudet. You can take a look at https://infra.apache.org/publishing-maven-artifacts.html for more information on the Apache Software Foundation (ASF) maven artifact publishing process. Summary: Release candidate artifacts are pushed to a staging area and can be promoted after the release vote passed.
Thanks for the links! I've been publishing to the Maven Central Repository, I know how that works.
It doesn't sound to me like they forbid publishing snapshots, just that it shouldn't be documented, which is weird, but whatever. It should be alright to deploy snapshots and keep it a "secret", no? They say we "should" do this and that, but if none of their services offers support for Maven artifacts, I suppose this means we can use something else, right?
As libgfortran has changed their ABI a few times over the years, you will need to include libgfortran.so in the jar (which we can distribute under AL2 License thanks to the GCC Runtime Library Exception). However, you must not include libquadmath.so (dependency of libgfortran.so) as it is GPL licensed.
Yes, that's not a problem. However, if we don't have libquadmath.so, libgfortran.so isn't going to load, so is it still useful?
For the gpu version mxnet-2.0-20201222.141246-19-linux-x86_64-gpu.jar, would it make sense to use cu110 instead of gpu if built with cuda 11.0 etc?
I guess? :) In any case, that's not a problem either. However, it's becoming increasingly irrelevant to try to support multiple versions of CUDA given their accelerating release cycle.
Yes, that's not a problem. However, if we don't have libquadmath.so, libgfortran.so isn't going to load, so is it still useful?
There haven't been any ABI breaks in libquadmath.so. Thus we can simply ask users to install libquadmath.so by themselves and everything will work. Our users will not be able to find an incompatible libquadmath.so
It doesn't sound to me like they forbid publishing snapshots, just that it shouldn't be documented, which is weird, but whatever. It should be alright to deploy snapshots and keep it a "secret", no?
They say we "should" do this and that, but if none of their services offers support for Maven artifacts, I suppose this means we can use something else, right?
Yes, that's not a problem. However, if we don't have libquadmath.so, libgfortran.so isn't going to load, so is it still useful?
There haven't been any ABI breaks in libquadmath.so. Thus we can simply ask users to install libquadmath.so by themselves and everything will work. Our users will not be able to find an incompatible libquadmath.so
I suppose it's more business friendly, but it's not a requirement for releasing binaries under Apache, correct? That is, this is a policy specific to the MXNet project?
It doesn't sound to me like they forbid publishing snapshots, just that it shouldn't be documented, which is weird, but whatever. It should be alright to deploy snapshots and keep it a "secret", no?
Yes, I saw that too. There's already a few snapshots from MXNet there, so I assume we can use it freely: http://repository.apache.org/content/groups/snapshots/org/apache/mxnet/
(Oh and look at that, the artifacts are using a generic "-gpu" string as part of their names. I would like to consult with the authors of those artifacts to see what they think of adding versions here before committing to tracking specific versions of CUDA. It's not something I would personally relish to do, until someone actually cares about it at least that is. The only person that ever complained about this for OpenCV is @EricXSun: bytedeco/javacpp-presets#918 )
They say we "should" do this and that, but if none of their services offers support for Maven artifacts, I suppose this means we can use something else, right?
I'm not sure what you mean.
I was referring to the snapshot repository, which they do offer, so we're good for that, but if we need something else, it would be good to know what the official stance is concerning the use of external services. I suppose anything from GitHub is OK, but other than that, I wonder.
I suppose it's more business friendly, but it's not a requirement for releasing binaries under Apache, correct? That is, this is a policy specific to the MXNet project?
It is a requirement to release the binaries under AL2. You can refer to https://www.apache.org/legal/resolved.html for a list of compatible and incompatible licenses. LGPL is Category-X (not allowed) as it places restrictions on the larger work.
Oh and look at that, the artifacts are using a generic "-gpu" string as part of their names. I would like to consult with the authors of those artifacts to see what they think of adding versions here before committing to tracking specific versions of CUDA. It's not something I would personally relish to do, until someone actually cares about it at least that is.
It's fine with me to re-use the existing pattern if others don't mind.
Now, where can I get an account for that server... Anyway, someone will just need to put their credentials as secrets for GitHub Actions and then we just need to change the URL here for the snapshots:
We can open a ticket with Apache Infra. Would you like to open a PR first?
it would be good to know what the official stance is concerning the use of external services. I suppose anything from GitHub is OK, but other than that, I wonder.
It's fine to use external services as long as the project maintainers (PPMC) control the usage and the published artifacts are compliant with the ASF polices (for example, don't contain LGPL compontents).
Ok, I've finally updated my fork accordingly along with a few additional changes: saudet@0966818
If that looks good enough for a pull request, I'll do that right away! @leezu
Activity
lanking520 commentedon Mar 6, 2020
I would propose Option 3. and 4.
DJL is a new Java framework that build on top of any engines. It brings the ease for Java developers to have close to numpy experience in Java. It introduced an interface that defined Java to train and run inference on different ML/DL models.
In the engine layer, we implemented the MXNet specific engine that allows users to achieve most of the up-to-date functionalities:
MXNet specific
DJL
Maintainance
With the benefit listed above, I would recommend Option 3 to go for the DJL path since it already covered most up-to-date MXNet feature and supporting all different symbolic/imperative training/inference.
For Option 4: I am also thinking of bring our JNA layer back to MXNet so the community can build up their own Java/Scala frontend if they don't like DJL.
terrytangyuan commentedon Mar 6, 2020
I propose option 1 and 2 since it took us a lot of efforts to bring MXNet to Scala originally and there are already adopters of Scala API in industries (some may not have been disclosed yet). But I am open to other options. Not familiar with DJL though but I assume @frankfliu and @lanking520 are the masters behind DJL.
gigasquid commentedon Mar 9, 2020
@lanking520 thanks for the clarification above. A further question - How do you envision a current Scala MXNet user migrate their code? Is it going to be mostly reusable or is it going to be a complete rewrite for them?
zachgk commentedon Mar 9, 2020
It is going to be closer to a complete rewrite. On the other hand, making a new Scala API would be imperative instead of symbolic and I think there are going to be a lot of operator changes to better match numpy in 2.0. I don't think the migration costs for a Scala 2.0 would be that much less anyway
For users who don't want a full rewrite, they can continue using an old release or whatever new releases we make on the v1.x branch.
gigasquid commentedon Mar 9, 2020
For the Clojure package. It is a lot easier to interop with Java than with Scala - so if the the base is Java that everything is using - it will be better for Clojure.
szha commentedon Mar 16, 2020
+1 for option 1 and 2. Also +1 for 4 as long as it doesn't add a dependency
My concerns on 3 and 4 are that DJL is a separate project which has its own release cycle. Having it to support MXNet's inference will cause delays as DJL upgrades to the latest version. This will also complicate the testing and validation.
Overall, I think a minimum set of API for at least inference is needed for MXNet JVM ecosystem users.
leezu commentedon Apr 13, 2020
Another data point is that all of our Scala tests fail randomly with
src/c_api/c_api_profile.cc:141: Check failed: !thread_profiling_data.calls_.empty():, so there seem to be some underlying issues.#17067
leezu commentedon Apr 24, 2020
Another data point is that we currently only support OpenJDK 8 but the JVM languages are broken with OpenJDK 11 which is used on Ubuntu 18.04 for example. See #18153
33 remaining items
leezu commentedon Dec 8, 2020
MXNet supports both cython and ctypes (fallback) for the Python interface. It depends on your build configuration.
https://github.com/apache/incubator-mxnet/blob/master/CMakeLists.txt#L91
We may want to change the default for MXNet 2
It's also fine to use Github Actions if that's easier for you. The main reason for using Jenkins is that the MXNet test suite is too large for a free service such as Github Actions and that there are also GPU tests involved. Java tests can initially run on Github Actions and be migrated later to Jenkins based on need.
saudet commentedon Dec 23, 2020
I've updated my fork with a workflow for Java build on GitHub Actions: saudet@2be0540
Please let me know what you think of this!
It's currently building and testing for Linux (CentOS 7), Mac, and Windows on x86_64 with and without CUDA:
https://github.com/saudet/incubator-mxnet/actions/runs/437312011
(It looks like the build doesn't work for CUDA 11.1 with Visual Studio 2019 yet, but that's unrelated to Java.)
Since my account at Sonatype doesn't have deploy access to
org.apache, the artifacts are getting deployed here for now:https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/2.0-SNAPSHOT/
But this can be changed by updating only a single line here:
https://github.com/saudet/incubator-mxnet/blob/add-javacpp/java/build.gradle#L8
In any case, the
javadocsecondary artifact also gets deployed as part of the build there.Where does the publishing to the main site happen? Somewhere in here by the looks of it:
https://github.com/apache/incubator-mxnet/tree/master/ci/publish/website
We can fetch the latest
javadocarchive this way, so I assume we could add that to the scripts?For that, GitHub Actions now support self-hosted runners, where we just need to provision some machines on the cloud somewhere, and install the equivalent of Jenkins Agent on them, and that's it. Much easier than maintaining Jenkins.
leezu commentedon Dec 23, 2020
Thank you @saudet. You can take a look at https://infra.apache.org/publishing-maven-artifacts.html for more information on the Apache Software Foundation (ASF) maven artifact publishing process. Summary: Release candidate artifacts are pushed to a staging area and can be promoted after the release vote passed.
One thing to note is that ASF policies do not allow publishing unreleased (nightly) artifacts to the general public. Those should be placed at special location and only used by interested community members. You can take a look at http://www.apache.org/legal/release-policy.html#publication and this FAQ entry http://www.apache.org/legal/release-policy.html#host-rc
Do you have any suggestion how to best handle it with your Github Actions script / Maven?
Github Actions isn't very mature yet. You can see in the doc that "Self-hosted runners on GitHub do not have guarantees around running in ephemeral clean virtual machines, and can be persistently compromised by untrusted code in a workflow." I don't think that's acceptable for projects accepting contributions from the general public.
leezu commentedon Dec 23, 2020
I downloaded the
mxnet-2.0-20201222.141246-19-linux-x86_64.jarand find thatAs
libgfortranhas changed their ABI a few times over the years, you will need to includelibgfortran.soin thejar(which we can distribute under AL2 License thanks to the GCC Runtime Library Exception). However, you must not includelibquadmath.so(dependency oflibgfortran.so) as it is GPL licensed.For the gpu version
mxnet-2.0-20201222.141246-19-linux-x86_64-gpu.jar, would it make sense to usecu110instead ofgpuif built with cuda 11.0 etc?marcoabreu commentedon Dec 23, 2020
Regarding security: I think that the quoted paragraph has the same (in)securities as our jenkins setup, doesn't it?
leezu commentedon Dec 23, 2020
I don't think so. Microsoft specifically says "We recommend that you do not use self-hosted runners with public repositories." It indicates to me that they have very little confidence in their security model. https://docs.github.com/en/free-pro-team@latest/actions/hosting-your-own-runners/about-self-hosted-runners#self-hosted-runner-security-with-public-repositories
marcoabreu commentedon Dec 23, 2020
Yes I think they are mentioning the same security problem we are having with our jenkins slaves. Any user could run arbitrary code and install a rootkit. Hence the separation towards restricted slaves.
So from that point of view, I don't consider the github actions self runner any less secure than our jenkins slaves. But of course still insecure.
leezu commentedon Dec 23, 2020
The problem with runners I had in mind is that there used to be no API to start new instances for each job, but rather that the instances had to be up and running all the time and would be re-used for all jobs. Thus any compromise would be truly persistent. We don't do that in our Jenkins setup, where instances are terminated time-to-time.
But I just checked the Github documentation and Microsoft team has resolved this issue and now provides an API that can provision new runners upon demand. So if there are volunteers, it should be fine to migrate to Github Actions. For example, https://040code.github.io/2020/05/25/scaling-selfhosted-action-runners
saudet commentedon Dec 25, 2020
Thanks for the links! I've been publishing to the Maven Central Repository, I know how that works.
It doesn't sound to me like they forbid publishing snapshots, just that it shouldn't be documented, which is weird, but whatever. It should be alright to deploy snapshots and keep it a "secret", no? They say we "should" do this and that, but if none of their services offers support for Maven artifacts, I suppose this means we can use something else, right?
Yes, that's not a problem. However, if we don't have
libquadmath.so,libgfortran.soisn't going to load, so is it still useful?I guess? :) In any case, that's not a problem either. However, it's becoming increasingly irrelevant to try to support multiple versions of CUDA given their accelerating release cycle.
saudet commentedon Dec 25, 2020
FWIW, it looks to me like libquadmath is LGPL, not GPL: https://github.com/gcc-mirror/gcc/blob/master/libquadmath/COPYING.LIB
leezu commentedon Dec 25, 2020
There haven't been any ABI breaks in libquadmath.so. Thus we can simply ask users to install libquadmath.so by themselves and everything will work. Our users will not be able to find an incompatible libquadmath.so
You're right, but the consequence is the same.
Yes. Reading the https://infra.apache.org/publishing-maven-artifacts.html again, there is also https://repository.apache.org/snapshots which may be the best location for snapshots?
I'm not sure what you mean.
saudet commentedon Jan 5, 2021
I suppose it's more business friendly, but it's not a requirement for releasing binaries under Apache, correct? That is, this is a policy specific to the MXNet project?
Yes, I saw that too. There's already a few snapshots from MXNet there, so I assume we can use it freely:
http://repository.apache.org/content/groups/snapshots/org/apache/mxnet/
(Oh and look at that, the artifacts are using a generic "-gpu" string as part of their names. I would like to consult with the authors of those artifacts to see what they think of adding versions here before committing to tracking specific versions of CUDA. It's not something I would personally relish to do, until someone actually cares about it at least that is. The only person that ever complained about this for OpenCV is @EricXSun: bytedeco/javacpp-presets#918 )
Now, where can I get an account for that server... Anyway, someone will just need to put their credentials as secrets for GitHub Actions and then we just need to change the URL here for the snapshots:
https://github.com/saudet/incubator-mxnet/blob/add-javacpp/java/build.gradle#L143
And that's it. They will appear exactly as they currently are on Sonatype:
https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/mxnet/2.0-SNAPSHOT/
I was referring to the snapshot repository, which they do offer, so we're good for that, but if we need something else, it would be good to know what the official stance is concerning the use of external services. I suppose anything from GitHub is OK, but other than that, I wonder.
leezu commentedon Jan 5, 2021
It is a requirement to release the binaries under AL2. You can refer to https://www.apache.org/legal/resolved.html for a list of compatible and incompatible licenses. LGPL is Category-X (not allowed) as it places restrictions on the larger work.
It's fine with me to re-use the existing pattern if others don't mind.
We can open a ticket with Apache Infra. Would you like to open a PR first?
It's fine to use external services as long as the project maintainers (PPMC) control the usage and the published artifacts are compliant with the ASF polices (for example, don't contain LGPL compontents).
saudet commentedon Jan 26, 2021
Ok, I've finally updated my fork accordingly along with a few additional changes: saudet@0966818
If that looks good enough for a pull request, I'll do that right away! @leezu
leezu commentedon Jan 26, 2021
Thank you. It's fine with me. Once you open the PR, @lanking520 and @gigasquid may be able to review too