Ruby/Policy 2.0

From ALT Linux Wiki
Stub.png
Sisyphus Policy draft
Authur(s) — Малъ Скрылевъ
Discussion in devel@
Discussed since 05.02.2019


Ruby Policy

Rules for ruby module and application packages

Introduction

Ruby and modularity

NOTE: To simplify link usage, they will be treated as for 2.6.1 ruby version.

The ruby language have the modularity feature begginging with the day-spring of its invention in 1993th, so if we have the core functions of ruby are placed into the libruby.so, are built-in part of the language, then ruby's standard library is modularized, and its modules are resided in the /usr/lib/ruby/ folder that has the interpreted modules inside the consequently named folders. For example: parser module is placed in the racc module: 

   /usr/lib/ruby/racc/parser.rb

Inside the folder named as <arch>-<os>, there are some compiled parts for the respective modules, so 'cparse modules is placed in racc module: 

   /usr/lib/ruby/x86_64-linux/racc/cparse.so

Therefore the when a side library (module) ought to be be used in ordinal ruby applications, then its library modules must be placed into the respective folders in /usr/lib/ruby/site_ruby/, and the last is the only place to reside libraries not from the ruby standard set.

The the modern context, which is valid for 2019, in order to deploy additional modules, the rubygems and bundler modules are used, which kind of the modules is called as gem, and the first of them previously treated as usual ruby side module. The rubygems has begun in 2009 Apr., bundler in four months last than rubygems in Aug the same year. In present, the pair of the gems is de-facto an de-jure the only dynamically working system of dependencies of ruby modules system.

In modern projects the ruby gem dependencies can be provided with two ways:

  • via <gem>.gemspec;
  • via Gemfile.

The .gemspec is the only file to declare dependencies for rubygems between the gems themselves. The dependencies in the file can be joined into two groups separately: runtime, and development, and the least can be used also to test the gem project.

The Gemfile is the file to define local dependencies (additionally to .gemspec) from the other gems. The project containing the Gemfile counkl not be exactly the gem, but it can be one of the modern ruby application type as: rails or sinatra application, ruby script application. The dependencies can be joined into various kinds of groups along the default one like production, development, test, assets etc. Based on the current environment and written in the Gemfile scheme, the bundler makes a decision itself, which the gems shall be loaded, and which must be hidden that leads to situation when explicit declaration to call a module from a gem will be avoided. For example it can be observed in rails application like chef or foreman, or script application opennebula.

Typical Gemfile can be found here.

Dependencies to ruby also can be defined in Gemfile or .gemspec.

Versioning

In the ruby and its packaging subsystem the semantic versioning approach is applied, in which the first number responds to the major modification in the module, saying when the API is changed. The second number - minor modification, for example when a new feature is added to the module, and the third number means the patch change.

Comparison to the previous policy

Strictly speaking the approach with dependencies search over the only require calls to the modern time is significantly restricted by the following points:

  • this method calls are not always exactly detecting the real dependencies list for gems, and increasingly for Gemfile-based projects, insofar as there may be non-mandatory calls to require in the code, which are framed in the begin-end code that makes the call optional;
  • for Gemfile-based projects the require calls are implicit and executed by bundler indirectly based on the Gemfile and .gemspec files analysis, if any, then bundler monitors for integrity of the current module development environment, this leads to that the require call to external module, in case the dependency to the module isn't declared in the Gemfile, will raises the module not found error that will block the project development (in case of rpm building to build error);
  • a name of external module is not always can be inferred from analysis of require call lines, since some modules have non-standard naming in a relation to their gem, for example a gem can have name 'foo-bar', its modules can have FooBar, FooBar, или Foo/Bar, but correspondence to those files will have foo_bar, foo/bar, and the last is falsely triggered on foo module name;
  • implicit loading of the external project modules, because bundler make the calls to that modules itself;
  • technique for dependency detection from require isn't capable to control API-version of the dependent package, when it is changed, especially in case when the package requires old and non-updated API.

Thus, in modern times is it more safely to use Gemfile/.gemspec file pair analysis to determine the dependencies.

The 'ruby' package

In the system there is the only libruby library, with a single set of the stdlibs modules.

When a new version of the libruby package has been built, all the stdlibs modules is rebuilt also, along with all gems that contain dynamically loaded .so libraries that conforms to the new version of libruby, or where the Gemfile/.gemspec file pair contains explicit requirement to the ruby version in.

External view of package

  1. packages that have been built from a gem, must have gem- prefix;
  2. source packages containing the rails, sinatra or similar applications can be without a prefix or have rubyapp- prefix;
  3. stdlibs modules along with other projects that don't belong to previous two points, and non-gem libraries must have ruby- prefix, and their files be placed into site_ruby folder;
  4. underscore chars "_" are replaced to minus '-' char in the names of packages.

Dependencies

  1. From now dependencies is being supervised based on Gemfile/.gemspec file pair analysis.
  2. Export of automatically calculated gem dependencies has view of gem(gemname) = version and rubygem(gemname) = version, the last form of the dependency has been kept to make them compatible to redhat ones.

Dependency conflict

In case of the dependency conflict is risen, one of the following ways to resolve them is applied:

  1. when and increase of a dependent gem version doesn't lead to degrade of workability of the parent gem, it is useful to apply explicit inline gem replacement by using gem_replace_version, for example from ~> 1.1.7 to ~> 1.7 for gem named "gem_name", you shall to specify the only target version of the depending gem:
%gem_replace_version gem_name ~> 1.7
  1. since bundler' allows keeping in the same tree of gems more than one version of each, but ALT packaging system doesn't allow making that thing, it is necessary to create an additional compatible package with the required version of the same gem. Name of the package must contain the same package name (as original non-compat), but also have full version including patch number fulled with "minus" chars. For example gem compat-gem will have full name gem-compat-gem-1-2-1.2.0.src.rpm;
  2. Gem dependencies, which there is building .so libraries from scources in, the dependency to current version of ruby is setup automatically, in order to rebuild of the ruby with a new version have forces to rebuild such kind of packages.

RPM Macros

Macros, which are required to gem and ruby-application packetizing, are defined in the rpm-build-ruby. And this file must be defined in spec with keyword BuilderRequires(pre), so we'll get BuildRequires(pre): rpm-build-ruby:

Ruby build macros are the following: 

%gemname                %(/bin/sed -e "s/\\(dbus\\|ldap\\)/ruby-\\1/" -e "s/^\\(ruby\\|gem\\)-//" -e "s/-\\(doc\\|devel\\)$//" -e "s/rcairo/cairo/" <<< %name)

%__ruby                 %_bindir/ruby
%__setup_rb             %_bindir/setup.rb
%bundle                 %_bindir/bundle

%ruby_func()            %__ruby -rrubygems -rrbconfig -e %*
%ruby_rubyconf_func()   %__ruby -rrubygems -rrbconfig -e 'print RbConfig::CONFIG["%*"]'

%ruby_gemspec           %(%ruby_func 'print File.join( Gem.dir, "specifications", "%gemname-%version.gemspec" )')
%ruby_gemspecdir        %(%ruby_func 'print File.join( Gem.dir, "specifications" )')
%ruby_gemextdir         %(%ruby_func 'print File.join( Gem.dir, "extensions", RbConfig::CONFIG["sitearch"], RbConfig::CONFIG["ruby_version"], "%gemname-%version" )')
%ruby_gemlibdir         %(%ruby_func 'print File.join( Gem.dir, "gems", "%gemname-%version" )')
%ruby_gemdocdir         %(%ruby_func 'print File.join( Gem.dir, "doc", "%gemname-%version" )')
%ruby_gemincludedir     %(%ruby_func 'print File.join( RbConfig::CONFIG["includedir"], "%gemname" )')
%ruby_includedir        %(%ruby_rubyconf_func includedir)')
%ruby_gemsextdir        %(%ruby_func 'print File.join( Gem.dir, "extensions", RbConfig::CONFIG["sitearch"], RbConfig::CONFIG["ruby_version"])')
%ruby_gemslibdir        %(%ruby_func 'print File.join( Gem.dir, "gems" )')
%ruby_gemsdocdir        %(%ruby_func 'print File.join( Gem.dir, "doc" )')
%ruby_ridir             %(%ruby_rubyconf_func ridir)

%gem_build              %__setup_rb build
%gem_install            %__setup_rb install --install_prefix=%buildroot
%gem_test               %__setup_rb test
%gem_show               %__setup_rb show --install_prefix=%buildroot

%rails_build            %__setup_rb build
%rails_install          %__setup_rb install --install_prefix=%buildroot
%rails_test             %__setup_rb test
%rails_show             %__setup_rb show --install_prefix=%buildroot
# env required macros
%_ruby_gem_rplc_list    %nil
%gem_replace_version()  %global _ruby_gem_rplc_list %_ruby_gem_rplc_list:%*

Files placement

The library files for a gem must be placed to the proper subfolders for the gem in /usr/lib/ruby/gems/2.5.0/, where 2.5.0 is the current ruby version, without a patch number.

Package specification notes

  1. all the packages containing the main code (meaning ruby and dynamic libraries) of the gem, must have the Development/Ruby group, and packages containing executables must have group name that belongs to its purpose.
  2. packages, which is not containing any kind of architecture-dependent binaries, must have a declaration of noarch architecture explicitly.

Typical .spec

To fix an old or broken packages, along with the to create new ones, the maintainer can use the following typical spec file: 

%define        pkgname gemmodule
Name:          gem-%pkgname
Version:       0.0.0
Release:       alt1
Summary:       <summary>
Group:         Development/Ruby
License:       <LICENSE>
URL:           https://site.org/
# VCS:         https://github.com/user/gemmodule.git
Source:        %name-%version.tar

BuildRequires(pre): rpm-build-ruby
BuildRequires: gem(another_gem_name)

%description
%summary


%package       -n %pkgname
Summary:       %summary
Group:         <required group>
BuildArch:     noarch

%description   -n %pkgname
%summary


%package       doc
Summary:       Documentation for %gemname gem
Group:         Development/Documentation
BuildArch:     noarch

%description   doc
Documentation for %gemname gem.


%package       devel
Summary:       Development files for %gemname gem
Group:         Development/Documentation
BuildArch:     noarch

%description   devel
Development files for %gemname gem.

%prep
%setup

%build
%gem_build

%install
%gem_install

%check
%gem_test

%files
%ruby_gemspec
%ruby_gemlibdir
%ruby_gemextdir

%files         -n %pkgname
%_bindir/*

%files         devel
%ruby_includedir/*

%files         doc
%ruby_gemdocdir

%changelog

The %gem_install macro is being expanded to call to setup.rb install action, and can change its behaviour when you shallpass to the command -s key, and specify the shebang explicitly, which it must use: 

%gem_install --shebang auto

it forces to automatically detect ruby interpreter installation path and replace all the ruby executables with the proper shebang line.

Testing

Testing of the ruby based packages can be carried out by the gem-setup package that defines and runs required to the testing of the packages specs, which are usually resided in the spec/test/features folders. The required the execute that test procedure gems are defined in the Gemspec/.gemspec pair. In order to enable the testing procedure we need to explicitly decleare the required gems in the package .spec, for example: 

BuildRequires: gem(required_to_test_gem) >= 1.0

This scheme migration rules

  1. all the Requires dependencies for all the subpackages are determined automatically, so that all the dependencies of suck kind must be rejected;
  2. BuildRequires dependencies for the package in case when testing shell be executed (see %check section), or when it is really necessary to do, must be explicitly defined, and can be got by the setup.rb application;
  3. when the package has been renamed, it is required to explicitly define the old package name in Provides and Obsoletes keywords;
  4. In the %prep all the macros pointing to ruby are rejected, in particular %update_setup_rb;
  5. In the %build all the macros pointing to ruby are replaced with %ruby_build, in particular %ruby_build, %ruby_config is begin replaced to %gem_build;
  6. In the %install all the macros pointing to ruby are replaced with %gem_install;
  7. In the %check all the macros pointing to ruby are replaced with %gem_test, although in case of the extremely and urgent need this macro can be grayed out;
  8. In the %files class sections the following rules of replacement are applied: %[ruby]gem_specdir to %ruby_gemspecdir, %ruby_sitelibdir to %ruby_gemlibdir, %ruby_sitearchdir to %ruby_gemextdir;
  9. In the %files for documentation %ruby_ri_sitedir is replaced with %ruby_gemdocdir.

Automatization

To automatize the watch and update the ruby-based packages the gem gem-rubobot shall be used, to accomplish the following functions:

  • execute the watch for new ruby-based package versions and notify the subscribers about their appearance on rubygems.org;
  • execute the watch for new version of ruby language, notify the subscribers about their appearance;
  • make an integrity control over the gem subsystem in Sisyphus, by the aim of the specific Gemfile;
  • monitor on the new non-packaged gem download count (from rubygems.org), and when the count is become more than a packetize threshold (by default 1M downloads), in automatic mode create an RPM-package based on the gem, and notify in both case of success or failure;
  • try to update gem packages in automatic mode keeping integrity control of the built gem packages, and notify when it is impossible;
  • try to update ruby application packages in automatic mode keeping integrity control of the built gem packages, the build here is allowing on-the-fly the replacement gem version to packages that are presented in the repo, and notify when it is impossible;
  • update gem packages packages in semi-automatic mode (i.e. by the directive from maintainer) keeping integrity control of the built gem packages, with an automatic creation of the gem compatible packages;
  • update ruby language package packages in semi-automatic mode (i.e. by the directive from maintainer) with automatic rebuild of dependent packages.