Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #185958 > unrolled thread

How to Keep Track of Changes to the System

Back to article view | Back to linux.debian.user

Contents

  How to Keep Track of Changes to the System ray <ray@aarden.us> - 2017-08-26 05:40 +0200
    Re: How to Keep Track of Changes to the System Andy Smith <andy@strugglers.net> - 2017-08-26 06:10 +0200
      Re: How to Keep Track of Changes to the System ray <ray@aarden.us> - 2017-08-26 15:10 +0200
    Re: How to Keep Track of Changes to the System Nemeth Gyorgy <friczy@freemail.hu> - 2017-08-26 11:50 +0200
      Re: How to Keep Track of Changes to the System ray <ray@aarden.us> - 2017-08-26 15:20 +0200
    Re: How to Keep Track of Changes to the System Dan Ritter <dsr@randomstring.org> - 2017-08-26 12:10 +0200
      Re: How to Keep Track of Changes to the System ray <ray@aarden.us> - 2017-08-26 15:30 +0200
        Re: How to Keep Track of Changes to the System Dan Ritter <dsr@randomstring.org> - 2017-08-26 19:00 +0200
        Re: How to Keep Track of Changes to the System Joe <joe@jretrading.com> - 2017-08-26 23:20 +0200
          Re: How to Keep Track of Changes to the System Mario Castelán Castro <marioxcc.MT@yandex.com> - 2017-08-27 17:20 +0200
    Re: How to Keep Track of Changes to the System "hdv@gmail" <hdv.jadev@gmail.com> - 2017-08-27 13:50 +0200
      Re: How to Keep Track of Changes to the System ray <ray@aarden.us> - 2017-08-31 05:00 +0200
        Re: How to Keep Track of Changes to the System "hdv@gmail" <hdv.jadev@gmail.com> - 2017-08-31 10:50 +0200
        Re: How to Keep Track of Changes to the System rhkramer@gmail.com - 2017-08-31 23:40 +0200
          Re: How to Keep Track of Changes to the System hdv@gmail <hdv.jadev@gmail.com> - 2017-09-01 07:50 +0200
            Re: How to Keep Track of Changes to the System rhkramer@gmail.com - 2017-09-01 13:40 +0200
    Re: How to Keep Track of Changes to the System rhkramer@gmail.com - 2017-08-27 16:00 +0200
      Re: How to Keep Track of Changes to the System ray <ray@aarden.us> - 2017-08-31 05:20 +0200
        Re: How to Keep Track of Changes to the System rhkramer@gmail.com - 2017-08-31 23:40 +0200
    Re: How to Keep Track of Changes to the System Mario Castelán Castro <marioxcc.MT@yandex.com> - 2017-08-27 17:50 +0200

#185958 — How to Keep Track of Changes to the System

I would like to find a way to keep track of changes I make to my system.  It seem that I may learn from others on how they keep track of changes they make to their systems.

When I make changes, I don't remember where I made changes or why.  

It would be great to have a log of what changes I've made, where they were made, how they were made (direct edit, scripted, etc.), why I made them, references that I used to determine the change, and what was the outcome.

Right now, I get lost in my documentation.  I research solutions, make notes in Onenote on a Windows machine, record configurations files that I will test.  But It is difficult to record results such as syslogs or console transactions.  More challenging is that I have different notebook tabs for different objectives.  So when I want to see what I changed, I have to go through many different objectives because I don't know what object I was shooting for when I made the change.

I would really like to hear how others track their changes or suggestions how I may tack changes.

I store all the changes on a different computer because I screw up the installation on my machine under test and rebuild the OS.  The laptop I am building to run Xen is on its 28th build.  

I would appreciate any suggestions.

Ray 

[toc] | [next] | [standalone]

#185960

Hi,

On Fri, Aug 25, 2017 at 08:14:52PM -0700, ray wrote:
> I would really like to hear how others track their changes or suggestions how I may tack changes.

I configure almost everything with configuration management like
puppet or ansible. Then the configuration is treated like code, can
be documented like code, stored in revision management (e.g. git)
like code.

I honestly don't think it's overkill for even one system, because as
you say, it is tricky to document things properly even for one
system.

I won't always go as far as installing and configuring a package
with the config management straight off. Depending on what system I
am working on I will sometimes "cheat" and install it manually with
apt, configure the files with an editor etc. But I do always at
least try to "go back" and recreate the working config with
config management so it's repeatable.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#185981

On Friday, August 25, 2017 at 11:10:04 PM UTC-5, Andy Smith wrote:
> Hi,
> 
...> 
> I won't always go as far as installing and configuring a package
> with the config management straight off. Depending on what system I
> am working on I will sometimes "cheat" and install it manually with
> apt, configure the files with an editor etc. But I do always at
> least try to "go back" and recreate the working config with
> config management so it's repeatable.

Andy,  Thank you.  I did not know these existed.  I am going to study these opportunities.

Ray

[toc] | [prev] | [next] | [standalone]

#185975

2017-08-26 05:14 keltezéssel, ray írta:
> I would like to find a way to keep track of changes I make to my system.  It seem that I may learn from others on how they keep track of changes they make to their systems.
>
> When I make changes, I don't remember where I made changes or why.  
>
> It would be great to have a log of what changes I've made, where they were made, how they were made (direct edit, scripted, etc.), why I made them, references that I used to determine the change, and what was the outcome.
>
> Right now, I get lost in my documentation.  I research solutions, make notes in Onenote on a Windows machine, record configurations files that I will test.  But It is difficult to record results such as syslogs or console transactions.  More challenging is that I have different notebook tabs for different objectives.  So when I want to see what I changed, I have to go through many different objectives because I don't know what object I was shooting for when I made the change.
>
> I would really like to hear how others track their changes or suggestions how I may tack changes.
>
> I store all the changes on a different computer because I screw up the installation on my machine under test and rebuild the OS.  The laptop I am building to run Xen is on its 28th build.  
>
> I would appreciate any suggestions.
I use etckeeper. Of course in tracks changes in /etc (and
subdirectories) only, but it is enough for me.

[toc] | [prev] | [next] | [standalone]

#185983

On Saturday, August 26, 2017 at 4:50:06 AM UTC-5, Nemeth Gyorgy wrote:
...
> I use etckeeper. Of course in tracks changes in /etc (and
> subdirectories) only, but it is enough for me.

Nemeth,

Thank you.  I like the idea of using a version control system.  

Ray

[toc] | [prev] | [next] | [standalone]

#185976

On Fri, Aug 25, 2017 at 08:14:52PM -0700, ray wrote:
> I would like to find a way to keep track of changes I make to my system.  It seem that I may learn from others on how they keep track of changes they make to their systems.
> 
> When I make changes, I don't remember where I made changes or why.  
> 
> It would be great to have a log of what changes I've made, where they were made, how they were made (direct edit, scripted, etc.), why I made them, references that I used to determine the change, and what was the outcome.


For a single system, etckeeper is an excellent choice. It stores
changes anywhere in /etc (and optionally in other places) in
git or subversion.

You can go a little farther easily by learning git directly.
https://try.github.io/levels/1/challenges/1  is a good resource.


If you have multiple systems that need to be handled in similar 
ways, you want to learn an advanced configuration management
system like chef, puppet, ansible, salt...

-dsr-

[toc] | [prev] | [next] | [standalone]

#185984

On Saturday, August 26, 2017 at 5:10:05 AM UTC-5, Dan Ritter wrote:
...
> For a single system, etckeeper is an excellent choice. It stores
> changes anywhere in /etc (and optionally in other places) in
> git or subversion.
> 
> You can go a little farther easily by learning git directly.
> https://try.github.io/levels/1/challenges/1  is a good resource.
> 
> If you have multiple systems that need to be handled in similar 
> ways, you want to learn an advanced configuration management
> system like chef, puppet, ansible, salt...
> 
> -dsr-

Dan,  

Thank you for the list of solutions.  It is interesting that SVN can be used with etckeeper.  It looks like I should learn git.  I have used SVN for other things, but I am easily pulled from my comfort zone for value.  

There is an interesting challenge here on where/how to keep repositories on a laptop.  It is valuable to have them locally as often my problems are networking; if the repositories are local, I can use another box to view them, but sometimes it may be a challenge to move files when connectivity is lost.  I am sure there is an architecture that will be suitable.  

It seems like there may be value in using both a config mgr and a version control system.  I will start checking into these to better understand.

Thank you,
Ray

[toc] | [prev] | [next] | [standalone]

#185991

On Sat, Aug 26, 2017 at 06:02:07AM -0700, ray wrote:
> On Saturday, August 26, 2017 at 5:10:05 AM UTC-5, Dan Ritter wrote:
> ...
> > For a single system, etckeeper is an excellent choice. It stores
> > changes anywhere in /etc (and optionally in other places) in
> > git or subversion.
> > 
> > You can go a little farther easily by learning git directly.
> > https://try.github.io/levels/1/challenges/1  is a good resource.
> > 
> > If you have multiple systems that need to be handled in similar 
> > ways, you want to learn an advanced configuration management
> > system like chef, puppet, ansible, salt...
> > 
> > -dsr-
> 
> Dan,  
> 
> Thank you for the list of solutions.  It is interesting that SVN can be used with etckeeper.  It looks like I should learn git.  I have used SVN for other things, but I am easily pulled from my comfort zone for value.  
> 
> There is an interesting challenge here on where/how to keep repositories on a laptop.  It is valuable to have them locally as often my problems are networking; if the repositories are local, I can use another box to view them, but sometimes it may be a challenge to move files when connectivity is lost.  I am sure there is an architecture that will be suitable.  

Sure. git is distributed; you can easily check it out on another
system, and thus have backups anytime you are connected to a
network, while having a complete local record as well.

> It seems like there may be value in using both a config mgr and a
version control system.  I will start checking into these to better
understand.

They are complementary solutions; config management solves "how
to get things in the right shape" problems, and version control
solves "what did this look like before?" problems.

-dsr-

[toc] | [prev] | [next] | [standalone]

#186004

On Sat, 26 Aug 2017 06:02:07 -0700 (PDT)
ray <ray@aarden.us> wrote:

 
> 
> Thank you for the list of solutions.  It is interesting that SVN can
> be used with etckeeper.  It looks like I should learn git.  I have
> used SVN for other things, but I am easily pulled from my comfort
> zone for value.  

Git is very widely used, and on important projects, so it is being
vigorously maintained. It's probably the right choice for new projects.
> 
> There is an interesting challenge here on where/how to keep
> repositories on a laptop.  It is valuable to have them locally as
> often my problems are networking; if the repositories are local, I
> can use another box to view them, but sometimes it may be a challenge
> to move files when connectivity is lost.  I am sure there is an
> architecture that will be suitable.  

Git creates a repository (by default) within the directory you base it
on, so copying the directory copies the repository. Git, both
command-line and GUI, exists for *nix and Windows, and a repository can
be operated from either. A backup of /etc to a USB stick, containing a
git repository, can be opened by git on another machine, and another
platform. I actually use git mostly on Windows, and mostly for its
'intended' use with software, though I also use it to track circuit
diagrams and PCB layouts under construction.

-- 
Joe

[toc] | [prev] | [next] | [standalone]

#186020

[Multipart message — attachments visible in raw view] — view raw

On 26/08/17 16:10, Joe wrote:
>> Thank you for the list of solutions.  It is interesting that SVN can
>> be used with etckeeper.  It looks like I should learn git.  I have
>> used SVN for other things, but I am easily pulled from my comfort
>> zone for value.  
> 
> Git is very widely used, and on important projects, so it is being
> vigorously maintained. It's probably the right choice for new projects.
New project should use Mercurial. Existing project should switch to
Mercurial.

Although Git can do anything that can be needed (and so can Mercurial),
the difference is that Git has an horrendously designed interface and
the concepts it is based on are many times irrational.

For example, consider the “git reset” command. This one deserves an
award for the most irrationally designed command in all of GNU/Linux.

If you want to change what commit the current branch points to, you must
use “git reset”, and you must memorize the meanings of “--soft”,
“--mixed”, “--hard” and “--keep”. If you do it wrong (e.g.: “keep”
instead of “--mixed”, you lose data).

But surprisingly, git reset does not always change what commit the
current branch points to. Sometimes it just moves files from a commit to
the index (“staging area”). (“git reset <COMMIT> -- .”).

So “git reset” does 4 things with little relation: Sometimes it moves
what commit the current branch points to and *maybe* changes the working
directory and staging area. Sometimes it just de-stages your staged
changes (“git reset”). Sometimes it cancels a failed merge. Sometimes it
copies files from an older commit to the staging area. These tasks are
group into a single command for no logical reason.

Now, this is no problem *after* you have learned how to use Git. You may
even think that well, it ought to be difficult to learn how to use it,
but that is incorrect. It is difficult only because the interface and
the concepts behind it are irrationally designed.

By contrast, Mercurial interface is very clean. Every command always
does one simple thing. This does not mean that it is less powerful.
Although in a previous time (years ago) when one could not do powerful
history modification as in Git, that era is now history.

Imagine how many man-hours are wasted learning how to use Git, which
would otherwise be used to do actual programming.

-- 
Do not eat animals, respect them as you respect people.
https://duckduckgo.com/?q=how+to+(become+OR+eat)+vegan

[toc] | [prev] | [next] | [standalone]

#186012

On 2017-08-26 05:14, ray wrote:
> I would like to find a way to keep track of changes I make to my system.  It
> seem that I may learn from others on how they keep track of changes they make
> to their systems.
> 
> When I make changes, I don't remember where I made changes or why.
> 
> It would be great to have a log of what changes I've made, where they were
> made, how they were made (direct edit, scripted, etc.), why I made them,
> references that I used to determine the change, and what was the outcome.
> 
> Right now, I get lost in my documentation.  I research solutions, make notes
> in Onenote on a Windows machine, record configurations files that I will
> test.  But It is difficult to record results such as syslogs or console
> transactions.  More challenging is that I have different notebook tabs for
> different objectives.  So when I want to see what I changed, I have to go
> through many different objectives because I don't know what object I was
> shooting for when I made the change.
> 
> I would really like to hear how others track their changes or suggestions how
> I may tack changes.
> 
> I store all the changes on a different computer because I screw up the
> installation on my machine under test and rebuild the OS.  The laptop I am
> building to run Xen is on its 28th build.
> 
> I would appreciate any suggestions.
> 
> Ray
> 

Hi Ray,

I just returned from a short holiday, so I am a bit late to the party, but... if
you don't want to set up a full versioning system I might have something else
for you. About 10 years ago I had the same need as you. What I did was write a
perl-script that automatically makes a timestamped backup of each file you edit
to a directory you define yourself (in that directory the full path of the
original is preserved). You use it like visudo, you just call it like this:

vicf <name of file to edit>

All the rest happens automagically.

Of course this will only help for plain-text files and it doesn't provide for
the annotations you mentioned. But if you are interested I
can mail it to you.

Grx HdV

P.S. Here's the output of the help so you can decide if this is what you need:

vicf --help

Usage:
    vicf [options] <argument>

    List of options:

    [-h|--help|-?] [--manual] [-V|--version] [-r|--root <directory>]
    [-d|--datetime <format>] [-n|--sequencenr <format>]
    [-a|--append_sequencenr] [-p|--permissions] [-o|--owner] [-g|--group]
    [-t|--times] [-b|--backup] [-x|--x_editor]

Options:
    --help
        Print a brief help message.

    --manual
        Print the full manual.

    --version
        Print version and copyright information.

    --root
        The directory under which a dated copy of the original file should
        be stored.

    --datetime
        A format string suitable for strftime(). Together with the local
        time this parameter will used as input for strftime(). The result
        will be appended to the name of the target file. See man 3 strftime
        for more details.

    --sequencenr
        A format string suitable for sprintf(). This will be used to
        generate a sequence number, which will be append to the filename if
        the target file already exists.

    --append_sequencenr
        Append a sequence number to the filename, even if it does not exist.
        Used to start sequences at 1 instead of 2.

    --permissions
        Preserve the access permissions of the original file.

    --owner
        Preserve the owner of the original file.

    --group
        Preserve the group of the original file.

    --times
        Preserve the access and modification times of the original file.

    --backup
        Instruct the editor to make a backup of the original file. This is a
        convenience option that has nothing to do with the dated copy of the
        original file.

    --x_editor
        Start the editor in graphical mode instead of console mode.

    At the top of the code there is a section named 'User-definable
    defaults' where defaults appropriate for the current environment can be
    set. Doing so alleviates the need to specify options on every invocation
    of the program.

Arguments:
    This program accepts only one argument, which is the path to the file to
    be edited.

[toc] | [prev] | [next] | [standalone]

#186215

On Sunday, August 27, 2017 at 6:50:06 AM UTC-5, hdv@gmail wrote:
> On 2017-08-26 05:14, ray wrote:
> > I would like to find a way to keep track of changes I make to my system.  ...snip
> Hi Ray,
> 
> I just returned from a short holiday, so I am a bit late to the party, but... if
> you don't want to set up a full versioning system I might have something else
> for you. About 10 years ago I had the same need as you. What I did was write a
> perl-script that automatically makes a timestamped backup of each file you edit
> to a directory you define yourself (in that directory the full path of the
> original is preserved). You use it like visudo, you just call it like this:
> 
> vicf <name of file to edit>
> 
> All the rest happens automagically.
> 
> Of course this will only help for plain-text files and it doesn't provide for
> the annotations you mentioned. But if you are interested I
> can mail it to you.
> 
> Grx HdV
...snip

Yes, I would like to work with this.  I should be able to modify the perl script to also save a tag file to hold the metadata.  So now I have a reason to learn some perl.

Thank you.
Ray

[toc] | [prev] | [next] | [standalone]

#186226

On 2017-08-31 04:39, ray wrote:
> On Sunday, August 27, 2017 at 6:50:06 AM UTC-5, hdv@gmail wrote:
>> On 2017-08-26 05:14, ray wrote:
>>> I would like to find a way to keep track of changes I make to my system.  ...snip
>> Hi Ray,
>>
>> I just returned from a short holiday, so I am a bit late to the party, but... if
>> you don't want to set up a full versioning system I might have something else
>> for you. About 10 years ago I had the same need as you. What I did was write a
>> perl-script that automatically makes a timestamped backup of each file you edit
>> to a directory you define yourself (in that directory the full path of the
>> original is preserved). You use it like visudo, you just call it like this:
>>
>> vicf <name of file to edit>
>>
>> All the rest happens automagically.
>>
>> Of course this will only help for plain-text files and it doesn't provide for
>> the annotations you mentioned. But if you are interested I
>> can mail it to you.
>>
>> Grx HdV
> ...snip
> 
> Yes, I would like to work with this.  I should be able to modify the perl script to also save a tag file to hold the metadata.  So now I have a reason to learn some perl.
> 
> Thank you.
> Ray

Here it is. Make sure to set the variables in the section "User-definable
defaults". I use the SWITCH statement to configure the script for use on
multiple systems. You might not need that. In that case just make sure you set
$root.

Good luck with it!

If you make any changes, please share them with the list. Maybe others might
find them useful too.

Grx HdV

===[begin script]====


#!/usr/bin/perl

#TODO : add support for settings stored in ~/.vicfrc or an explicitly given file

our $VERSION = '0.92';

use strict;
use warnings;
use Getopt::Long qw(:config bundling);
use Pod::Usage;
use Sys::Hostname;
use Cwd qw(realpath getcwd);
use POSIX qw(strftime sysconf _PC_CHOWN_RESTRICTED);
use File::Spec;
use File::Copy;

my $hostname = hostname();

################################################################################
#User-definable defaults
################################################################################

my $root = '';
SWITCH: {
  if ($hostname eq '') {
    $root = '';
    last SWITCH;
  }
  if ($hostname eq '') {
    $root = '';
    last SWITCH;
  }
  if ($hostname eq '') {
    $root = '';
    last SWITCH;
  }
}

my $datetime_format = '-%Y%m%d';                   #d
my $sequencenr_format = '-%02d';                   #n
my $append_sequencenr = 1;                         #a
my $keep_permissions = 1;                          #p
my $keep_owner = 1;                                #o
my $keep_group = 1;                                #g
my $keep_times = 1;                                #t
my $backup_file = 0;                               #b
my $x_editor = 0;                                  #x
my $editor_path = '/usr/bin/vim';
my $x_editor_path = '/usr/bin/gvim';
my $backup_option = '-c "set backup"';
my $no_backup_option = '-c "set nobackup"';

################################################################################
#Internal variables
################################################################################

#Defaults for commandline options
my $help = 0;
my $manual = 0;
my $show_version = 0;
my $debug = 0;
my $verbose = 0;

################################################################################
#Parse the commandline arguments
################################################################################

#Get all options
GetOptions(#Standard options
           'debug|D+', \$debug,
           'help!', \$help,
           'h|?', \$help,
           'manual!', \$manual,
           'version!', \$show_version,
           'V', \$show_version,
           'verbose|v+', \$verbose,
           #Options specific for this program
           'root|r=s', \$root,
           'datetime|d=s', \$datetime_format,
           'sequencenr|n=s', \$sequencenr_format,
           'append_sequencenr!', \$append_sequencenr,
           'a!', \$append_sequencenr,
           'permissions!', \$keep_permissions,
           'p!', \$keep_permissions,
           'owner!', \$keep_owner,
           'o!', \$keep_owner,
           'group!', \$keep_group,
           'g!', \$keep_group,
           'times!', \$keep_times,
           't!', \$keep_times,
           'backup!', \$backup_file,
           'b!', \$backup_file,
           'x_editor!', \$x_editor,
           'x!', \$x_editor
          ) or pod2usage(0);
pod2usage(verbose => 1, exitval => 0) if $help;
pod2usage(verbose => 2, exitval => 0) if $manual;
if ($show_version) {
  print "vicf version $VERSION (c) 2015 Jadev\n";
  exit 0;
}

#Assign the first non-option argument to a variable for easier use
die "No file to be edited was given.\n" unless $ARGV[0];
my $source = $ARGV[0];

#Sanitize paths for easier use
$root = expand_path($root);
die "Path pointing to the repository ($root) is invalid.\n" unless $root;
$source = expand_path($source);
die "Path pointing to the file to be edited ($source) is invalid.\n" unless $source;

#Check validity of given options
die "The given root directory does not exist.\n" unless -d $root;
die "The datetime formatstring may contain only alphanumeric or punctuation
characters.\n"
  unless $datetime_format =~ /^[[:print:]]*$/;    #May be empty
die "The sequencenumber formatstring may contain only alphanumeric or
punctuation characters.\n"
  unless $sequencenr_format =~ /^[[:print:]]*$/;    #May be empty

#Verify validity of the given arguments
#I know the file tests could be combined, but this allows for specific messages
#and performance isn't an issue here anyway.
die "The file to be edited does not exist.\n" unless -e $source;
die "The file to be edited is not a file.\n" unless -f $source;
die "The file to be edited is not readable.\n" unless -r $source;
die "The file to be edited is not writable.\n" unless -w $source;
warn "You can edit only one file at a time. Superfluous arguments will be
ignored.\n"
  if @ARGV > 1;

#Print option values and arguments
if ($debug) {
  warn '-' x 80, "\n";
  warn "[Options]\n";
  warn "root                   = $root\n";
  warn "timestamp_format       = $datetime_format\n";
  warn "sequencenr_format      = $sequencenr_format\n";
  warn "append_sequencenr      = $append_sequencenr\n";
  warn "keep_permissions       = $keep_permissions\n";
  warn "keep_owner             = $keep_owner\n";
  warn "keep_group             = $keep_group\n";
  warn "keep_times             = $keep_times\n";
  warn "backup_file            = $backup_file\n";
  warn "x_editor               = $x_editor\n";
  warn "\n";
  warn "[Arguments]\n";
  if (@ARGV) {
    warn join(', ', @ARGV), "\n";
  } else {
    warn "No arguments\n";
  }
  warn "\n";
  warn "[Other variables]\n";
	warn "hostname               = $hostname\n";
  warn "source                 = $source\n";
  warn '-' x 80, "\n";
  #exit 0;
}

################################################################################
#Subroutines
################################################################################

#Expand a given path to its absolute equivalent
sub expand_path {
  my $path = shift;
  print "[DEBUG] Expanding path   $path ...\n" if $debug;
  #Get current working directory
  my $cwd = getcwd();
  #Get directory separator (getcwd() returns absolute paths!)
  my $sep = substr($cwd, 0, 1);
  #If this path is not absolute, make it so
  if ($path !~ /^$sep/) {
    if (($sep eq '/') && ($path =~ /^~/)) {
      #Path is relative to user's home directory (Unix specific)
      #TODO : find out if there's a better way to determine if we're running on
      #a Unix platform. Using $^O won't do, because you'd have to match all
      #names of platforms that are (not) Unix-like
      $path =~ s{ ^ ~ ( [^/]* ) } { $1 ? ( getpwnam($1))[7] || return undef
                                       : ( $ENV{HOME} || $ENV{LOGDIR} ||
                                           (getpwuid($>))[7] ) }ex;
    }
    elsif (not File::Spec->file_name_is_absolute($path)) {
      #Path is relative to current working directory. Remember: File::Spec
      #thinks tilde paths are relative too!
      $path = File::Spec->catdir($cwd, $path);
    }
  }
  #Resolve symlinks, embedded . and .. and superfluous file separators
  #Remember: realpath() does not need to be fed an existing path,
  #so this subroutine can be used to expand new paths too
  $path = realpath($path) or undef;
  print "[DEBUG] Expanded path is $path\n" if $debug;
  return $path;
}

#Get the current timestamp, based on the local time
sub get_timestamp {
  return strftime($datetime_format, localtime(time));
}

#Get the full path to the destination file
sub get_destination {
  my $source = shift; #Full path to the source file
  my ($src_vol, $src_dir, $src_file) = File::Spec->splitpath($source);
  my ($root_vol, $root_dir, $root_file) = File::Spec->splitpath($root, 1);
  my $dest_dir = File::Spec->catdir($root_dir, $src_dir);
  my $dest_file = $src_file . get_timestamp();
  my $destination = File::Spec->catpath($root_vol, $dest_dir, $dest_file);
  my $count = 1;
  my $tmp_dest;
  if ($append_sequencenr) {
    $tmp_dest = $destination . sprintf($sequencenr_format, $count);
  } else {
    $tmp_dest = $destination;
  }
  while (-e $tmp_dest) {
    $count++;
    $tmp_dest = $destination . sprintf($sequencenr_format, $count);
  }
  return $tmp_dest;
}

#Recursively create directories
sub make_dir {
  my $src = shift; #Full path to the source file
  my $dest = shift; #Full path to the destination file
  my ($src_vol, $src_dir, $src_file) = File::Spec->splitpath($src);
  my ($dest_vol, $dest_dir, $dest_file) = File::Spec->splitpath($dest);
  my ($root_vol, $root_dir, $root_file) = File::Spec->splitpath($root, 1);
  #Recursively step through the destination path
  my $current_src_dir = '';
  my $current_dest_dir = $root_dir;
  my $current_src_path;
  my $current_dest_path;
  print '[DEBUG] Result of splitdir ('
    . scalar(File::Spec->splitdir($src_dir)) . ') = ['
    . join(':', File::Spec->splitdir($src_dir)) . "]\n"
    if $debug;
  foreach my $sub_dir (File::Spec->splitdir($src_dir)) {
    $current_src_dir = File::Spec->catdir($current_src_dir, $sub_dir);
    $current_dest_dir = File::Spec->catdir($current_dest_dir, $sub_dir);
    $current_src_path = File::Spec->catpath($src_vol, $current_src_dir, '');
    $current_dest_path = File::Spec->catpath($dest_vol, $current_dest_dir, '');
    next if $sub_dir eq ''; #splitpath returns empty values for the first and
                            #last array entries
    next if -d $current_dest_path;
    if ($debug) {
      print "[DEBUG] Making destination directory $current_dest_path ($sub_dir)
...\n";
    } else {
      mkdir($current_dest_path) or die "Cannot create directory: $!\n";
    }
    copy_attributes($current_src_path, $current_dest_path);
  }
}

#Copy file to its destination
sub copy_file {
  my $src = shift; #Full path to the source file
  my $dest = shift; #Full path to the destination file
  #Store original access time of source file (this is ugly, I know...)
  my $atime_src_file = (stat($src))[8];
  if ($debug) {
    print "[DEBUG] Copying source file $src to destination file $dest ...\n";
    print "[DEBUG] Original atime source file = " . strftime("%Y-%m-%d
%H:%M:%S", localtime($atime_src_file)) . "\n";
  } else {
    copy($src, $dest)
      or die "Could not copy file from $src to $dest: $!\n";
  }
  copy_attributes($src, $dest, $atime_src_file);
}

#Copy mode, owner and group attributes
sub copy_attributes {
  my $src = shift; #Full path to the source (can be a file or directory)
  my $dest = shift; #Full path to the destination (can be a file or directory)
  my $original_atime = shift; #Original atime of source, if source is a file
  my ($mode, $owner, $group, $atime, $mtime, $ctime) = (stat($src))[2, 4, 5, 8, 9];
  $atime = $original_atime if $original_atime;
  my $mode_str = sprintf("%04o", $mode & 07777); #Strip the file type bit
  my $owner_name = getpwuid $owner;
  my $group_name = getgrgid $group;
  my $atime_str = strftime("%Y-%m-%d %H:%M:%S", localtime($atime));
  my $mtime_str = strftime("%Y-%m-%d %H:%M:%S", localtime($mtime));
  if ($debug) {
    print "[DEBUG] Copying attributes from $src to $dest ...\n";
    print "[DEBUG] Setting mode of $dest to $mode_str ...\n" if $keep_permissions;
    print "[DEBUG] Setting owner of $dest to $owner_name ...\n" if $keep_owner;
    print "[DEBUG] Setting group of $dest to $group_name ...\n" if $keep_group;
    if ($keep_times) {
      print "[DEBUG] Setting atime of $dest to $atime_str ...\n";
      print "[DEBUG] Setting mtime of $dest to $mtime_str ...\n";
    }
  } else {
    #TODO : Make preserving attributes more portable
    chmod $mode & 07777, $dest
      or die "Could not set mode of $dest to $mode_str: $!\n"
      if $keep_permissions;
    if (not sysconf(_PC_CHOWN_RESTRICTED)) {
      chown $owner, -1, $dest
        or die "Could not set owner of $dest to $owner_name: $!\n"
        if $keep_owner;
    } else {
      chown $owner, -1, $dest
        or warn "The system won't allow you (" . scalar(getpwent()) . ") to
change ownership of $dest\n"
        if $keep_owner;
    }
    chown -1, $group, $dest
      or die "Could not set group of $dest to $group_name: $!\n"
      if $keep_group;
    #TODO : the access time of the source file is not always preserved (even
    #       though $atime is correct)!
    #TODO : print "[DEBUG] atime_str = " . strftime("%Y-%m-%d %H:%M:%S",
localtime($atime)) . "\n";
    utime $atime, $mtime, $dest
      or warn "Could not set access and modification times for $dest: $!\n"
      if $keep_times;
  }
}

#Open a file with vim
sub edit_file {
  my $file = shift; #Full path to the file to be edited
  my $editor;
  my $options;
  if ($x_editor) {
    $editor = $x_editor_path;
  } else {
    $editor = $editor_path;
  }
  if ($backup_file) {
    $options = $backup_option;
  } else {
    $options = $no_backup_option;
  }
  if ($debug) {
    print '[DEBUG] Opening source file with ' . $editor . ' ' . $options . ' ' .
$file . " ... \n";
  } else {
    system $editor, $options, $file;
  }
}

################################################################################
#Main program
################################################################################

#Define the path to which the original will be copied
my $destination = get_destination($source);

#Make sure the target directory exists
make_dir($source, $destination);

#Copy the original
copy_file($source, $destination);

#Edit the original
edit_file($source);

#Exit the program gracefully
exit 0;

__END__

################################################################################
#Documentation
################################################################################

=head1 NAME

vicf - edit a configuration file and keep a dated copy of the original.

=head1 SYNOPSIS

vicf [options] <argument>

List of options:

[-h|--help|-?] [--manual] [-V|--version] [-r|--root <directory>]
[-d|--datetime <format>] [-n|--sequencenr <format>] [-a|--append_sequencenr]
[-p|--permissions] [-o|--owner] [-g|--group] [-t|--times] [-b|--backup]
[-x|--x_editor]

=head1 OPTIONS

=over 4

=item B<--help>

Print a brief help message.

=item B<--manual>

Print the full manual.

=item B<--version>

Print version and copyright information.

=item B<--root>

The directory under which a dated copy of the original file should be stored.

=item B<--datetime>

A format string suitable for strftime(). Together with the local time this
parameter will used as input for strftime(). The result will be appended to the
name of the target file. See man 3 strftime for more details.

=item B<--sequencenr>

A format string suitable for sprintf(). This will be used to generate a
sequence number, which will be append to the filename if the target file
already exists.

=item B<--append_sequencenr>

Append a sequence number to the filename, even if it does not exist. Used to
start sequences at 1 instead of 2.

=item B<--permissions>

Preserve the access permissions of the original file.

=item B<--owner>

Preserve the owner of the original file.

=item B<--group>

Preserve the group of the original file.

=item B<--times>

Preserve the access and modification times of the original file.

=item B<--backup>

Instruct the editor to make a backup of the original file. This is a
convenience option that has nothing to do with the dated copy of the original
file.

=item B<--x_editor>

Start the editor in graphical mode instead of console mode.

=back

At the top of the code there is a section named 'User-definable defaults' where
defaults appropriate for the current environment can be set. Doing so
alleviates the need to specify options on every invocation of the program.

=head1 ARGUMENTS

This program accepts only one argument, which is the path to the file to be
edited.

=head1 DESCRIPTION

This program makes versioned copies of the files you edit. It is useful for
everyone who wants to have a history of changes made to their files, but
doesn't want to run a CVS, SubVersion or equivalent server just for that
purpose. A copy of the original file is stored in a repository, the root
directory, using the full path of the original file. A timestamp and optionally
a sequence number is appended to the file's name to make sure it is unique.
After that the file is opened with the preferred editor to be edited, just like
usual.

=head1 NOTES/TODO

Find out if there's a better way to determine if we're running on a Unix
platform than what is used now (first char of absolute path is a slash). Just
using $^O won't do either, because you'd have to match all names of platforms
that are or are not Unix-like

Make preserving access modes more portable (for use on non-Unix platforms).

Add support for storing settings in ~/.vicfrc or in an explicitly given
configuration file.

=head1 PREREQUISITES

=over 4

=item - Perl 5.005 or later

=back

=head1 HISTORY

=over 4

=item B<0.01 (2007-05-08)> - First public release.

=item B<0.02 (2007-05-20)> - Improved support for non-absolute paths.

=item B<0.92> (2015-11-21)> - Resolved bug in expand_path routine.

=back

=head1 BUGS

Somehow the access time of the original is not always preserved in the copy.
Need to find out what is causing this. In those cases where this happens the
atime given to utime() seems to be correct.

Let me know if you find another one (or more...).

=head1 AUTHOR

J.A. de Vries <hdv@jadev.org>

Current contact information and the master copy of this program can be found
at http://www.jadev.org

=head1 COPYRIGHT

Copyright (C) 2007 Jadev

This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 2 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
details.

You can find a copy of the GNU General Public License at
http://www.gnu.org/licenses/gpl.html

[toc] | [prev] | [next] | [standalone]

#186254

On Wednesday, August 30, 2017 10:39:38 PM ray wrote:
> ...snip
> 
> Yes, I would like to work with this.  I should be able to modify the perl
> script to also save a tag file to hold the metadata.  So now I have a
> reason to learn some perl.

I'm interested as well, if you could send a copy to me.  

@Ray: And even more so, Ray, after you modify it.

[toc] | [prev] | [next] | [standalone]

#186267

On 2017-08-31 23:36, rhkramer@gmail.com wrote:
> On Wednesday, August 30, 2017 10:39:38 PM ray wrote:
>> ...snip
>>
>> Yes, I would like to work with this.  I should be able to modify the perl
>> script to also save a tag file to hold the metadata.  So now I have a
>> reason to learn some perl.
> 
> I'm interested as well, if you could send a copy to me.  
> 
> @Ray: And even more so, Ray, after you modify it.
> 

Hi rhkramer,

I sent it to this list yesterday. If you missed it I can send it again. Just let
me know.

Grx HdV

[toc] | [prev] | [next] | [standalone]

#186275

On Friday, September 01, 2017 01:40:33 AM hdv@gmail wrote:
> On 2017-08-31 23:36, rhkramer@gmail.com wrote:
> > I'm interested as well, if you could send a copy to me.
> > 
> > @Ray: And even more so, Ray, after you modify it.

> I sent it to this list yesterday. If you missed it I can send it again.
> Just let me know.

Thanks, I saw it and saved it!

I also looked at it briefly (i.e., skimmed)--Perl is not my thing (if anything 
is) so, at some point, I will have to spend some time understanding how to use 
it--I suppose at the very least, put it in a file and mark it executable, and 
then invoke it with the file name.

I don't expect to have trouble with that, but also am busy with some other 
things for the next couple weeks, so not sure when I'll get to that--when I 
do, if I have questions, I have your email address.

Thanks again,
Randy Kramer

[toc] | [prev] | [next] | [standalone]

#186013

On Friday, August 25, 2017 11:14:52 PM ray wrote:
> I would like to find a way to keep track of changes I make to my system. 
> It seem that I may learn from others on how they keep track of changes
> they make to their systems.
> 
> When I make changes, I don't remember where I made changes or why.
> 
> It would be great to have a log of what changes I've made, where they were
> made, how they were made (direct edit, scripted, etc.), why I made them,
> references that I used to determine the change, and what was the outcome.
> 
> Right now, I get lost in my documentation.  I research solutions, make
> notes in Onenote on a Windows machine, record configurations files that I
> will test.  But It is difficult to record results such as syslogs or
> console transactions.  More challenging is that I have different notebook
> tabs for different objectives.  So when I want to see what I changed, I
> have to go through many different objectives because I don't know what
> object I was shooting for when I made the change.
> 
> I would really like to hear how others track their changes or suggestions
> how I may tack changes.
> 
> I store all the changes on a different computer because I screw up the
> installation on my machine under test and rebuild the OS.  The laptop I am
> building to run Xen is on its 28th build.
> 
> I would appreciate any suggestions.

In general, I have similar problems, and, I don't have a good general solution 
or suggestion at this time, but I do have one suggestion to address one 
specific part of the problem.

I would suggest that you create a separate partition on the "subject" computer 
(the one on which you rebuild the OS when you have a (significant) problem with 
the OS.  Then:
   * mount that partition each (or almost each) time you rebuild the OS 
   * never reformat or delete that partition
   * when you mount the partition, create a mounting point outside the FHS 
hierarchy (i.e., not in /home)--I typically create several such partitions 
(for different reasons) using my initials or similar, for instance, I have 
rhk01, rhk02, ..., back01, back02 and such
   * keep the change log (whatever form it takes on such a partition--with 
luck they will always be available, and, it is easier to c&p to a file on the 
computer in question rather than "transcribe" stuff to a different computer.

Of course, a similar result could be obtained by mounting a partion on some 
other computer over the network (using, for example, NFS or Samba), or using 
SSL to write notes on the subject computer but store them on a "safer" 
computer.

Of course, in any case, the files on that separate partition should be backed 
up regulalrly, just in case (and, particularly, just before you start an OS 
rebuild).

<the following is a little more general about the general approach I take (and 
tools I use), but my  mashup (read on) is not ready for prime time (or 
distribution to others, except maybe to someone really motivated who might 
also be interested in doing some programming to carry things forward---I guess 
I'm sort of saying "read the following at your own risk" ;-) >

With respect to an actual general change log (as opposed to where to keep it), 
I keep my notes in some specially formatted files for which I've created some 
"programming" (e.g., macros, and similar)  to create something that I call (at 
various times): my askSam workalike, my offline wiki-like thing, and some other 
similar names that I don't recall at the moment.  (Oh, I also consider it a 
"mashup" as it (each iteration) uses a variety of programs to provide all the 
features I want to have (for example, recoll for full text indexed search).

I've gone through several iterations, each using an editor with special macros 
to make things more convenient, but I don't currently have any features that 
do anything like monitor all of the files in /etc (and/or /var/log or similar 
locations) and record any changes, the date, and prompt me to enter a reason 
for the change.

That would be nice!

(Previous iterations have been based on nedit as the editor, then kate as the 
editor (with a different feature set, partially based on the capabilties of the 
editor), and have also included a full text indexed search system (using 
recoll).  My current focus (which I've been "working on" sporadically over the 
last 12 years or so, will be based on scintilla, which because scintilla is 
used as the editing "widgit" for quite a few editors, would open the door to 
letting me choose from a variety of editors.  (And then, maybe (not something 
I really considered before today), might include something like monitoring 
/home, etc. for changes (as mentioned somewhere above) (using things like--
well, I can't remember the name, but there are tools that monitor files for 
changes, and scintilla-based editors have a pretty powerful scripting language 
in Lua.

[toc] | [prev] | [next] | [standalone]

#186216

On Sunday, August 27, 2017 at 9:00:05 AM UTC-5, rhkr...@gmail.com wrote:
...snip...

Hi RHKR,

Thank you.  This is an interesting way to store the broken system.  It will be like a junk yard that I can copy out of.  

Partitioning will be a challenge.  Currently, this is laptop runs LVM.  I have two groups.  The system is on one, the other I plan on using for VM storage.  I could break the one for VMs into groups for file systems.  I am not sure how to use this.  Once I have these different groups, if I want to rebuild, I will need to have the new system installed to one of the unused VGs.  I have a  quandary about this each time I rebuild, I have a challenge with the Debian installer on where to put the new system.  It seem like I rebuild the LVM each time which means I would wipe out the previous system.

Ray

Ray

[toc] | [prev] | [next] | [standalone]

#186253

On Wednesday, August 30, 2017 10:56:44 PM ray wrote:
> Thank you.  This is an interesting way to store the broken system.  It will
> be like a junk yard that I can copy out of.
> 
> Partitioning will be a challenge.  Currently, this is laptop runs LVM.  I
> have two groups.  The system is on one, the other I plan on using for VM
> storage.  I could break the one for VMs into groups for file systems.  I
> am not sure how to use this.  Once I have these different groups, if I
> want to rebuild, I will need to have the new system installed to one of
> the unused VGs.  I have a  quandary about this each time I rebuild, I have
> a challenge with the Debian installer on where to put the new system.  It
> seem like I rebuild the LVM each time which means I would wipe out the
> previous system.

You're welcome!

I don't / have never used LVM, but I thought that at least in some scenarios, 
you created partitions and then installed LVM somehow on top of those 
partitions.  I assume that, if you create one more partition, you could just 
leave it out of LVM?

Maybe someone else can advise you.

[toc] | [prev] | [next] | [standalone]

#186022

[Multipart message — attachments visible in raw view] — view raw

On 25/08/17 22:14, ray wrote:
> I would like to find a way to keep track of changes I make to my system.  It seem that I may learn from others on how they keep track of changes they make to their systems.

I have a plain-text file of notes, which I keep under Mercurial version
control. I make a note here whenever I make any big change.

For manually installed packages, I install them under a directory in
“~/local/stow”. For example “~/local/stow/emacs”. Since there is a
one-to-one correspondence between packages and directories under the
“stow” directory, obtaining a list of what packages I have installed is
as easy as “ls ~/local/stow”.

The search path for executables is “~/local/bin”. I use GNU Stow to
automatically make symbolic links from here to the corresponding
directory under “~/local/stow”.

I can recommend GNU Stow to have better control over *manually*
installed packages. A common problem is that ones does “make install”
and then when one wants to delete the package, one does not know what
files one should delete, and ones does not realize if something is being
overwritten. GNU Stow solves that.

-- 
Do not eat animals, respect them as you respect people.
https://duckduckgo.com/?q=how+to+(become+OR+eat)+vegan

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web