Differences

This shows you the differences between two versions of the page.

--- it-tricks [2023/03/14 15:25]
vodrazka [TAR archive mounting]
+++ it-tricks [2024/03/05 18:04]
popel [LaTeX, pdf]
@@ Line 10: / Line 10: @@
   * **emacs**: ?
   * **nano**:
-  * **atom**: Martin Popel
   * **IntelliJ IDEA**: Jonáš Vidra
   * **Kate**: Jonáš Vidra
-  * **Kile** (offline TeX editing): Anša Vernerová
+  * **PyCharm**:
-  * **PyCharm**: Petr Bělohlávek
   * **VS Code** (remote access, IDE - Python, C++, etc. completion, terminals): Peter Polák
@@ Line 89: / Line 87: @@
 ===== LaTeX, pdf =====
-  * see also [[tex::triky]] and [[internal:publikace#collaborative-writing]]
+  * see also [[tex::triky]], [[internal:publikace#collaborative-writing]] and [[internal:thesis]]
   * [[http://mg.readthedocs.io/latexmk.html|latexmk]] can be used with ''-pdf -pvc'', so whenever you save a ''tex'' file, the corresponding ''pdf'' will be regenerated (and your pdf viewer will refresh). Other useful options are ''-interaction=nonstopmode -synctex=1'', see e.g. [[https://tex.stackexchange.com/questions/118489/what-exactly-is-synctex|synctex]]
   * [[https://www.pdflabs.com/docs/pdftk-cli-examples/|pdftk]] for merging and splitting pdf files (and much more) from the command line
@@ Line 100: / Line 98: @@
 ===== TAR archive mounting =====
-If you work with data consisting of many small files, you **should** store them in tar archives to save inodes. There is a way to mount a tar archive to a specific directory in the simmilar way you mount ISO image. You can use this command to //mount// (read-only) ARCHIVE to DIRECTORY:
+If you work with data consisting of many small files, you **should** store them in tar archives to save inodes and use LUSTRE filesystem efficiently. There is a way to mount a tar archive to a specific directory in a similar way you would mount an ISO image. You can use this command to //mount// (read-only) ARCHIVE (possibly compressed) to DIRECTORY:
   /opt/bin/tarmount ARCHIVE DIRECTORY
@@ Line 108: / Line 106: @@
   /opt/bin/tarmount -u DIRECTORY
-Run the command with //--help// option to see instructions for more advanced usage. The original tool can be obtained here [[https://github.com/mxmlnkn/ratarmount | ratarmount]]
+If you mount your data in this way, there is only one //open// operation instead of thousands or millions of such operations when opening each file separately. This will positively affect runtime when your data is stored on the LUSTRE filesystem.
+Run the command with //--help// option to see instructions for more advanced usage. The original tool can be obtained [[https://github.com/mxmlnkn/ratarmount | here]].
+It can be installed as [[https://appimage.org | AppImage]].
+==== Practical usage notes ====
+  - ''tarmount'' is based on FUSE. You need to do call the ''tarmount'' command on every node where you need to access the tarfile contents. If two jobs are running on the same node both of them will see the mounted content.
+  - if you finish your work you should always call ''tarmount -u DIRECTORY''. If you don't the directory will not become blocked for any subsequent attempts to ''tarmount'' anything on them and may be in a weird state indicated by many '?'s in output of the ''ls -al'' command. This can be solved by calling simple ''umount DIRECTORY''
+  - The ''tarmount'' command creates a database in the directory where the ''ARCHIVE'' is stored. The database is not automatically deleted by ''tarmount -u''. It is however reused by any subsequent ''tarmount'' command called on the same ''ARCHIVE''. This may speed up access for all jobs using the same ''tarmount''ed archive across the cluster.

[ Back to the navigation ] [ Back to the content ]

Institute of Formal and Applied Linguistics Wiki

Differences