Compare commits

...

13 Commits

Author SHA1 Message Date
DataHearth 0c8d52f32d
v0.4.0 2022-06-16 18:41:39 +02:00
DataHearth 6f24de2bd0
chore(release): change script permissions 2022-06-16 18:41:19 +02:00
DataHearth 5f52ce6658
chore(release): automate release creation 2022-06-15 12:04:15 +02:00
DataHearth e92265791e
chore(doc): update README.md 2022-06-14 13:23:04 +02:00
DataHearth ea25075c82
fix(save): remove folder before copy (avoid unwanted files) 2022-06-14 12:15:24 +02:00
DataHearth 56da676e15
chore(doc): update function documentation 2022-06-14 12:03:53 +02:00
DataHearth a0662e9066
fix(config): don't throw error when file not available on OS 2022-06-14 11:31:35 +02:00
DataHearth d3bab447e0
feat(cli): add verbose flag and a spinner for pkgs 2022-06-06 15:32:01 +02:00
DataHearth 588e3c1e66
chore(doc): update readme issues 2022-06-06 14:52:39 +02:00
DataHearth 555d05d640 chore(doc): update readme 2022-06-02 19:53:41 +02:00
DataHearth 46249857f3 feat(config): add SSH capability with user/pass or key/pass 2022-06-02 19:44:51 +02:00
DataHearth f4bf8070ac chore(git): add .env to .gitignore 2022-06-02 19:44:07 +02:00
DataHearth 3a7210bc69 chore(doc): update readme 2022-06-02 18:02:33 +02:00
18 changed files with 554 additions and 165 deletions

View File

@ -0,0 +1,31 @@
<a name="{{ (index .Versions 0).Tag.Name }}"></a>
## {{ if (index .Versions 0).Tag.Previous }}[{{ (index .Versions 0).Tag.Name }}]{{ else }}{{ (index .Versions 0).Tag.Name }}{{ end }} - {{ datetime "2006-01-02" (index .Versions 0).Tag.Date }}
{{ range (index .Versions 0).CommitGroups -}}
### {{ .Title }}
{{ range .Commits -}}
- {{ if .Scope }}**{{ .Scope }}:** {{ end }}{{ .Subject }}
{{ end }}
{{ end -}}
{{- if (index .Versions 0).RevertCommits -}}
### Reverts
{{ range (index .Versions 0).RevertCommits -}}
- {{ .Revert.Header }}
{{ end }}
{{ end -}}
{{- if (index .Versions 0).MergeCommits -}}
### Pull Requests
{{ range (index .Versions 0).MergeCommits -}}
- {{ .Header }}
{{ end }}
{{ end -}}
{{- if (index .Versions 0).NoteGroups -}}
{{ range (index .Versions 0).NoteGroups -}}
### {{ .Title }}
{{ range .Notes }}
{{ .Body }}
{{ end }}
{{ end -}}
{{ end -}}

View File

@ -1,8 +0,0 @@
.config-mapper.yml
.config-mapper.yml.template
.gitignore
LICENSE
README.md
CHANGELOG.md
.chglog
build

3
.gitignore vendored
View File

@ -1,3 +1,4 @@
.config-mapper.yml
build
.DS_Store
.DS_Store
.env

View File

@ -3,6 +3,17 @@
## [Unreleased]
<a name="v0.4.0"></a>
## [v0.4.0] - 2022-06-16
### Bug Fixes
- **config:** don't throw error when file not available on OS
- **save:** remove folder before copy (avoid unwanted files)
### Features
- **cli:** add verbose flag and a spinner for pkgs
- **config:** add SSH capability with user/pass or key/pass
<a name="v0.3.0"></a>
## [v0.3.0] - 2022-06-01
### Features
@ -60,6 +71,7 @@
- **config:** add yaml tags for yaml.v3
[Unreleased]: https://github.com/DataHearth/config-mapper/compare/v0.3.0...HEAD
[Unreleased]: https://github.com/DataHearth/config-mapper/compare/v0.4.0...HEAD
[v0.4.0]: https://github.com/DataHearth/config-mapper/compare/v0.3.0...v0.4.0
[v0.3.0]: https://github.com/DataHearth/config-mapper/compare/v0.2.0...v0.3.0
[v0.2.0]: https://github.com/DataHearth/config-mapper/compare/v0.1.0...v0.2.0

View File

@ -1,14 +0,0 @@
VERSION 0.6
FROM golang:1.18-alpine3.15
WORKDIR /config-mapper
build-macos:
COPY . .
RUN GOOS=darwin go build -o build/config-mapper main.go
SAVE ARTIFACT build/config-mapper /config-mapper AS LOCAL build/x86-x64_darwin_config-mapper
build-linux:
COPY . .
RUN GOOS=linux go build -o build/config-mapper main.go
SAVE ARTIFACT build/config-mapper /config-mapper AS LOCAL build/x86-x64_linux_config-mapper

View File

@ -1,7 +1,7 @@
# config-mapper
`config-mapper` is CLI utility tool to help you manage your configuration between systems.
It provides a set of tools to load your configuration from a system, save it on a git repository and then save it to a new system. This configuration can be a set of files, folders or even dependencies.
`config-mapper` is CLI utility tool to help you manage your configuration between UNIX systems.
It provides a set of tools to load your configuration from a system, save it into a git repository and then save it to a new system. This configuration can be a set of files, folders or even dependencies.
## Usage
@ -13,12 +13,36 @@ The system is detected automatically. You just need to specify whether the relat
You can get a configuration template [here](https://raw.githubusercontent.com/DataHearth/config-mapper/main/.config-mapper.yml.template).
### Installation
Using a pre-build binary:
- `wget`
```bash
wget https://github.com/DataHearth/config-mapper/releases/download/{RELEASE}/x86-x64_{linux|darwin}_config-mapper -O $HOME/.local/bin/
```
- `gh`
```bash
gh release download -r DataHearth/config-mapper {RELEASE} -d $HOME/.local/bin/ -p "x86-x64_{linux|darwin}_config-mapper"
```
Building from source:
```bash
git clone git@github.com:datahearth/config-mapper.git
cd config-mapper
go build -o $HOME/.local/bin/config-mapper
```
### Setup
Create a file called `.config-mapper.yml` in your `home` directory (it is the default search path for config-mapper).
If you wish to move it to another directory, you can have to choice to inform the tool. By either set an environment like this one: `CONFIG_MAPPER_CFG=/path/to/config/.config-mapper.yml`. Or by providing the `-c /path/to/config/.config-mapper.yml` flag to the tool.
If you wish to move it to another directory, you can choose by either setting an environment: `CONFIG_MAPPER_CFG=/path/to/config/.config-mapper.yml` or by using the `-c /path/to/config/.config-mapper.yml` flag.
Once the configuratio file created, run this command to initialize the repository localy:
Once the configuration file created, run this command to initialize the repository locally:
```bash
config-mapper init
@ -56,7 +80,7 @@ config-mapper save
All defined files and folders will be copied inside your repository.
If you want to exclude one part of your configuration file (files, folders), you can use these flags to ignore them `--disable-files` `--disable-folders`. Note, package managers are disable by default. You can enable this option using the `--pkgs` flag.
If you want to exclude one part of your configuration file (files, folders), you can use these flags to ignore them `--disable-files` `--disable-folders`. Note, package managers are disable by default. You can enable this option using the `--pkgs` flag.
You can also exclude files and folders from a given directory with a `.gitignore` like file named `.ignore`. Put it in the root directory of an included folder and add relative path to exclude (does not support glob for now). E.g:
@ -73,6 +97,7 @@ drwxr-xr-x - antoine 1 Jun 20:27 └── foo
```
`.ignore` content:
```
# bar file will be ignored
foo/bar
@ -81,13 +106,13 @@ foo/bar
egg
```
If `homebrew` is provided in the `installation-order` (default: `["apt", "homebrew"]`), it will override the `homebrew` field with all user installed packages (`brew leaves --installed-on-request`). The same principle will be implemented with `aptitude`.
If `homebrew` is provided in the `installation-order` (default: `["apt", "homebrew"]`), it will override the `homebrew` field with all user installed packages (`brew leaves --installed-on-request`). The same principle will be implemented with `Advanced Package Tool`.
template for your configuration:
```yaml
# NOTE: the $LOCATION if refering to the "storage.location" path. It'll be replaced automatically
# The left part of ":" is your repository location and right part when it should be on your system
# The left part of ":" is your repository location and right part is on your system
files:
- darwin: "$LOCATION/macos/.zshrc:~/.zshrc"
linux: "$LOCATION/linux/.zshrc:~/.zshrc"
@ -130,10 +155,9 @@ The same ignore flags are used in the `save` command.
## TO-DO
- [X] add `.ignore` file to ignore content inside directory
- [x] add `.ignore` file to ignore content inside directory
- [x] use remote configuration: SSH
- [ ] optimisation over speed and memory
- [ ] load configuration though SSH
- [ ] save configuration though SSH
- add more storage options
- [ ] smb storage
- [ ] nfs storage
@ -145,3 +169,7 @@ The same ignore flags are used in the `save` command.
Resolved by create a new primary key based on GitHub new GIT SSH standards ([issue](https://github.com/go-git/go-git/issues/411))
- Cloning from GitHub with `https BasicAuth` and 2FA activated: `authentication required`
Resolved by creating an access token and set it as password in configuration
- WSL might have a rough time with opened files by `homebrew` and throwing `Error: too many open files`.
[This thread](https://github.com/Homebrew/linuxbrew-core/issues/21139) discuss about this issue.
The workaround seems to be increase the filesystem limit (`ulimit -Hn && ulimit -Sn`). Another way is to launch again your command as homebrew already installed
some the packages.

View File

@ -7,6 +7,8 @@ import (
"time"
mapper "github.com/datahearth/config-mapper/internal"
"github.com/datahearth/config-mapper/internal/configuration"
"github.com/datahearth/config-mapper/internal/git"
"github.com/fatih/color"
"github.com/spf13/cobra"
"github.com/spf13/viper"
@ -22,7 +24,7 @@ var rootCmd = &cobra.Command{
Short: "Manage your systems configuration",
Long: `config-mapper aims to help you manage your configurations between systems
with a single configuration file.`,
Version: "v0.3.0",
Version: "v0.4.0",
}
var initCmd = &cobra.Command{
Use: "init",
@ -47,30 +49,38 @@ var saveCmd = &cobra.Command{
}
func init() {
cobra.OnInitialize(mapper.InitConfig)
cobra.OnInitialize(configuration.InitConfig)
rootCmd.AddCommand(initCmd)
rootCmd.AddCommand(loadCmd)
rootCmd.AddCommand(saveCmd)
rootCmd.PersistentFlags().BoolP("verbose", "v", false, "STDOUT will be more verbose")
rootCmd.PersistentFlags().StringP("configuration-file", "c", "", "location of configuration file")
rootCmd.PersistentFlags().String("ssh-user", "", "SSH username to retrieve configuration file")
rootCmd.PersistentFlags().String("ssh-password", "", "SSH password to retrieve configuration file")
rootCmd.PersistentFlags().String("ssh-key", "", "SSH key to retrieve configuration file (if a passphrase is needed, use the \"CONFIG_MAPPER_PASS\" env variable")
viper.BindPFlag("verbose", rootCmd.PersistentFlags().Lookup("verbose"))
viper.BindPFlag("configuration-file", rootCmd.PersistentFlags().Lookup("configuration-file"))
viper.BindPFlag("ssh-user", rootCmd.PersistentFlags().Lookup("ssh-user"))
viper.BindPFlag("ssh-password", rootCmd.PersistentFlags().Lookup("ssh-password"))
viper.BindPFlag("ssh-key", rootCmd.PersistentFlags().Lookup("ssh-key"))
loadCmd.PersistentFlags().Bool("disable-files", false, "files will be ignored")
loadCmd.PersistentFlags().Bool("disable-folders", false, "folders will be ignored")
loadCmd.PersistentFlags().Bool("pkgs", false, "packages will be installed")
viper.BindPFlag("load-disable-files", loadCmd.PersistentFlags().Lookup("disable-files"))
viper.BindPFlag("load-disable-folders", loadCmd.PersistentFlags().Lookup("disable-folders"))
viper.BindPFlag("load-enable-pkgs", loadCmd.PersistentFlags().Lookup("pkgs"))
loadCmd.Flags().Bool("disable-files", false, "files will be ignored")
loadCmd.Flags().Bool("disable-folders", false, "folders will be ignored")
loadCmd.Flags().Bool("pkgs", false, "packages will be installed")
viper.BindPFlag("load-disable-files", loadCmd.Flags().Lookup("disable-files"))
viper.BindPFlag("load-disable-folders", loadCmd.Flags().Lookup("disable-folders"))
viper.BindPFlag("load-enable-pkgs", loadCmd.Flags().Lookup("pkgs"))
saveCmd.PersistentFlags().Bool("disable-files", false, "files will be ignored")
saveCmd.PersistentFlags().Bool("disable-folders", false, "folders will be ignored")
saveCmd.PersistentFlags().Bool("pkgs", false, "packages will be saved")
saveCmd.Flags().Bool("disable-files", false, "files will be ignored")
saveCmd.Flags().Bool("disable-folders", false, "folders will be ignored")
saveCmd.Flags().Bool("pkgs", false, "packages will be saved")
saveCmd.Flags().BoolP("push", "p", false, "new configurations will be committed and pushed")
saveCmd.Flags().StringP("message", "m", strconv.FormatInt(time.Now().Unix(), 10), "combined with --push to set a commit message")
saveCmd.Flags().Bool("disable-index", false, "configuration index will not be updated")
viper.BindPFlag("save-disable-files", saveCmd.PersistentFlags().Lookup("disable-files"))
viper.BindPFlag("save-disable-folders", saveCmd.PersistentFlags().Lookup("disable-folders"))
viper.BindPFlag("save-enable-pkgs", saveCmd.PersistentFlags().Lookup("pkgs"))
viper.BindPFlag("save-disable-files", saveCmd.Flags().Lookup("disable-files"))
viper.BindPFlag("save-disable-folders", saveCmd.Flags().Lookup("disable-folders"))
viper.BindPFlag("save-enable-pkgs", saveCmd.Flags().Lookup("pkgs"))
viper.BindPFlag("push", saveCmd.Flags().Lookup("push"))
viper.BindPFlag("disable-index-update", saveCmd.Flags().Lookup("disable-index"))
viper.BindPFlag("message", saveCmd.Flags().Lookup("message"))
@ -84,7 +94,7 @@ func Execute() {
}
func save(cmd *cobra.Command, args []string) {
var c mapper.Configuration
var c configuration.Configuration
if err := viper.Unmarshal(&c); err != nil {
mapper.PrintError("failed to decode configuration: %v\n", err)
os.Exit(1)
@ -96,7 +106,7 @@ func save(cmd *cobra.Command, args []string) {
os.Exit(1)
}
r, err := mapper.NewRepository(c.Storage.Git, c.Storage.Path)
r, err := git.NewRepository(c.Storage.Git, c.Storage.Path)
if err != nil {
mapper.PrintError("failed to open repository at %s: %v\n", c.Storage.Path, err)
os.Exit(1)
@ -138,7 +148,7 @@ func save(cmd *cobra.Command, args []string) {
}
func load(cmd *cobra.Command, args []string) {
var c mapper.Configuration
var c configuration.Configuration
if err := viper.Unmarshal(&c); err != nil {
mapper.PrintError("failed to decode configuration: %v\n", err)
os.Exit(1)
@ -150,7 +160,7 @@ func load(cmd *cobra.Command, args []string) {
os.Exit(1)
}
r, err := mapper.NewRepository(c.Storage.Git, c.Storage.Path)
r, err := git.NewRepository(c.Storage.Git, c.Storage.Path)
if err != nil {
mapper.PrintError("failed to open repository at %s: %v\n", c.Storage.Path, err)
os.Exit(1)
@ -176,16 +186,15 @@ func load(cmd *cobra.Command, args []string) {
}
func initCommand(cmd *cobra.Command, args []string) {
var config mapper.Configuration
if err := viper.Unmarshal(&config); err != nil {
var c configuration.Configuration
if err := viper.Unmarshal(&c); err != nil {
mapper.PrintError("failed to decode configuration: %v\n", err)
os.Exit(1)
}
logger.Println("initializing config-mapper folder from configuration...")
if _, err := mapper.NewRepository(config.Storage.Git, config.Storage.Path); err != nil {
if _, err := git.NewRepository(c.Storage.Git, c.Storage.Path); err != nil {
mapper.PrintError("failed to initialize folder: %v\n", err)
os.Exit(1)
}

5
go.mod
View File

@ -3,9 +3,12 @@ module github.com/datahearth/config-mapper
go 1.17
require (
github.com/fatih/color v1.13.0
github.com/gernest/wow v0.1.0
github.com/go-git/go-git/v5 v5.4.2
github.com/spf13/cobra v1.3.0
github.com/spf13/viper v1.10.1
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
)
@ -14,7 +17,6 @@ require (
github.com/ProtonMail/go-crypto v0.0.0-20210428141323-04723f9f07d7 // indirect
github.com/acomagu/bufpipe v1.0.3 // indirect
github.com/emirpasic/gods v1.12.0 // indirect
github.com/fatih/color v1.13.0
github.com/fsnotify/fsnotify v1.5.1 // indirect
github.com/go-git/gcfg v1.5.0 // indirect
github.com/go-git/go-billy/v5 v5.3.1 // indirect
@ -36,7 +38,6 @@ require (
github.com/spf13/pflag v1.0.5 // indirect
github.com/subosito/gotenv v1.2.0 // indirect
github.com/xanzy/ssh-agent v0.3.0 // indirect
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 // indirect
golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d // indirect
golang.org/x/sys v0.0.0-20211210111614-af8b64212486 // indirect
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect

4
go.sum
View File

@ -123,6 +123,8 @@ github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYF
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI=
github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=
github.com/gernest/wow v0.1.0 h1:g9xdwCwP0+xgVYlA2sopI0gZHqXe7HjI/7/LykG4fks=
github.com/gernest/wow v0.1.0/go.mod h1:dEPabJRi5BneI1Nev1VWo0ZlcTWibHWp43qxKms4elY=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gliderlabs/ssh v0.2.2 h1:6zsha5zo/TWhRhwqCD3+EarCAgZ2yN28ipRnGPnwkI0=
github.com/gliderlabs/ssh v0.2.2/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
@ -407,6 +409,7 @@ go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9i
go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190103213133-ff983b9c42bc/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190219172222-a4c6cb3142f2/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
@ -531,6 +534,7 @@ golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190116161447-11f53e031339/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

View File

@ -0,0 +1,67 @@
package configuration
import (
"fmt"
"log"
"os"
"path"
"strings"
"github.com/fatih/color"
"github.com/spf13/viper"
)
var errLogger = log.New(os.Stderr, "", 0)
func InitConfig() {
h, err := os.UserHomeDir()
if err != nil {
errLogger.Fatalln(err)
}
if c := viper.GetString("configuration-file"); c != "" {
if strings.Contains(c, "ssh://") {
viper.AddConfigPath(h)
viper.SetConfigType("yml")
viper.SetConfigName(".config-mapper")
if err := loadConfigSSH(c); err != nil {
errLogger.Fatalln(err)
}
return
}
if strings.Contains(c, ".yml") {
viper.AddConfigPath(path.Dir(c))
} else {
viper.AddConfigPath(c)
}
}
if c := os.Getenv("CONFIG_MAPPER_CFG"); c != "" {
if strings.Contains(c, ".yml") {
viper.AddConfigPath(path.Dir(c))
} else {
viper.AddConfigPath(c)
}
}
viper.AddConfigPath(h)
viper.SetConfigType("yml")
viper.SetConfigName(".config-mapper")
viper.SetDefault("storage.location", fmt.Sprintf("%s/config-mapper", os.TempDir()))
viper.SetDefault("package-managers.installation-order", []string{"apt", "homebrew"})
if err := viper.ReadInConfig(); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
color.Error.Write([]byte(color.RedString("no configuration file found: %v\n", err)))
} else {
color.Error.Write([]byte(color.RedString("failed to read config: %v\n", err)))
}
os.Exit(1)
}
viper.Set("configuration-file", viper.ConfigFileUsed())
}

View File

@ -1,13 +1,4 @@
package mapper
import (
"fmt"
"os"
"path"
"strings"
"github.com/spf13/viper"
)
package configuration
type Configuration struct {
Storage Storage `mapstructure:"storage" yaml:"storage"`
@ -49,45 +40,3 @@ type PkgManagers struct {
Homebrew []string `mapstructure:"homebrew" yaml:"homebrew"`
Aptitude []string `mapstructure:"apt-get" yaml:"apt-get"`
}
func InitConfig() {
h, err := os.UserHomeDir()
if err != nil {
errLogger.Printf("can't get home directory through $HOME variable: %v\n", err)
os.Exit(1)
}
if c := viper.GetString("configuration-file"); c != "" {
if strings.Contains(c, ".yml") {
viper.AddConfigPath(path.Dir(c))
} else {
viper.AddConfigPath(c)
}
}
if c := os.Getenv("CONFIG_MAPPER_CFG"); c != "" {
if strings.Contains(c, ".yml") {
viper.AddConfigPath(path.Dir(c))
} else {
viper.AddConfigPath(c)
}
}
viper.AddConfigPath(h)
viper.SetConfigType("yml")
viper.SetConfigName(".config-mapper")
viper.SetDefault("storage.location", fmt.Sprintf("%s/config-mapper", os.TempDir()))
viper.SetDefault("package-managers.installation-order", []string{"apt", "homebrew"})
if err := viper.ReadInConfig(); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
PrintError(err.Error())
} else {
PrintError("failed to read config: %v\n", err)
}
os.Exit(1)
}
viper.Set("configuration-file", viper.ConfigFileUsed())
}

View File

@ -0,0 +1,175 @@
package configuration
import (
"bytes"
"errors"
"fmt"
"io/ioutil"
"os"
osUser "os/user"
"strings"
"github.com/fatih/color"
"github.com/spf13/viper"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/knownhosts"
)
func loadConfigSSH(uri string) error {
config, host, path, err := getSSHConfig(uri)
if err != nil {
return err
}
c, err := ssh.Dial("tcp", host, config)
if err != nil {
return err
}
defer c.Close()
s, err := c.NewSession()
if err != nil {
return err
}
defer s.Close()
buff := new(bytes.Buffer)
s.Stdout = buff
if err := s.Run(fmt.Sprintf("cat %s", path)); err != nil {
return err
}
if err := viper.ReadConfig(buff); err != nil {
return err
}
return nil
}
func getSSHConfig(uriFlag string) (*ssh.ClientConfig, string, string, error) {
var err error
var user, passwd, host, configPath, key string
uri := strings.Split(uriFlag, "ssh://")[1]
if key = viper.GetString("ssh-key"); key != "" {
uri, user, passwd, err = getCredentials(uri)
if err != nil {
return nil, "", "", err
}
host, configPath, err = getUriContent(uri)
if err != nil {
return nil, "", "", err
}
} else if user = viper.GetString("ssh-user"); user != "" {
host, configPath, err = getUriContent(uri)
if err != nil {
return nil, "", "", err
}
passwd = viper.GetString("ssh-password")
} else {
uri, user, passwd, err = getCredentials(uri)
if err != nil {
return nil, "", "", err
}
if passwd == "" {
passwd = viper.GetString("ssh-password")
}
host, configPath, err = getUriContent(uri)
if err != nil {
return nil, "", "", err
}
}
if user == "" {
color.Yellow("WARNING: no user was found in either the URI and flags. Current user will be used")
var currentUser *osUser.User
currentUser, err = osUser.Current()
if err != nil {
return nil, "", "", err
}
user = currentUser.Username
}
var auth ssh.AuthMethod
if key != "" {
auth, err = createPubKeyAuth(key)
if err != nil {
return nil, "", "", err
}
} else {
auth = ssh.Password(passwd)
}
h, err := os.UserHomeDir()
if err != nil {
return nil, "", "", err
}
hostKeyCallback, err := knownhosts.New(fmt.Sprintf("%s/.ssh/known_hosts", h))
if err != nil {
return nil, "", "", err
}
if len(strings.SplitN(host, ":", 1)) == 1 {
host = fmt.Sprintf("%s:22", host)
}
return &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{auth},
HostKeyCallback: hostKeyCallback,
}, host, configPath, nil
}
// getCredentials takes an SSH URI and returns (splitted URI, user, passwd, error)
//
// "passwd" can be empty in case of a single credential in URI
func getCredentials(uri string) (string, string, string, error) {
uriContent := strings.SplitN(uri, "@", 2)
if len(uriContent) == 1 {
fmt.Printf("uriContent: %v\n", uriContent)
return "", "", "", errors.New("no credentials in URI")
}
credentials := strings.SplitN(uriContent[0], ":", 2)
if len(credentials) == 1 {
return uriContent[1], credentials[0], "", nil
}
return uriContent[1], credentials[0], credentials[1], nil
}
// getUriContent takes an SSH URI and returns (host, path, error)
func getUriContent(uri string) (string, string, error) {
uriContent := strings.Split(uri, ":")
if len(uriContent) < 2 {
return "", "", errors.New("ssh URI is malformed. It's missing either a host or path. E.g: \"ssh://localhost:/my/config/file.yml\"")
}
return uriContent[0], uriContent[1], nil
}
func createPubKeyAuth(key string) (ssh.AuthMethod, error) {
var signer ssh.Signer
privateKey, err := ioutil.ReadFile(key)
if err != nil {
return nil, err
}
if passphrase := os.Getenv("CONFIG_MAPPER_SSH_PASSPHRASE"); passphrase != "" {
signer, err = ssh.ParsePrivateKeyWithPassphrase(privateKey, []byte(passphrase))
} else {
signer, err = ssh.ParsePrivateKey(privateKey)
}
if err != nil {
return nil, err
}
return ssh.PublicKeys(signer), nil
}

View File

@ -1,4 +1,4 @@
package mapper
package git
import (
"errors"
@ -6,6 +6,8 @@ import (
"os/exec"
"time"
"github.com/datahearth/config-mapper/internal/configuration"
"github.com/datahearth/config-mapper/internal/misc"
"github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/plumbing/object"
"github.com/go-git/go-git/v5/plumbing/transport"
@ -14,8 +16,7 @@ import (
)
var (
ErrDirIsFile = errors.New("path is a file")
ErrInvalidEnv = errors.New("found invalid environment variable in path")
ErrDirIsFile = errors.New("path is a file")
)
type RepositoryActions interface {
@ -38,18 +39,18 @@ type author struct {
email string
}
func NewRepository(config Git, repoPath string) (RepositoryActions, error) {
func NewRepository(config configuration.Git, repoPath string) (RepositoryActions, error) {
var auth transport.AuthMethod
if config.URL == "" {
return nil, errors.New("a repository URI is needed (either using GIT protocol or HTTPS)")
}
repoPath, err := absolutePath(repoPath)
repoPath, err := misc.AbsolutePath(repoPath)
if err != nil {
return nil, err
}
if config.SSH.Passphrase != "" && config.SSH.PrivateKey != "" {
privateKey, err := absolutePath(config.SSH.PrivateKey)
privateKey, err := misc.AbsolutePath(config.SSH.PrivateKey)
if err != nil {
return nil, err
}

View File

@ -5,6 +5,8 @@ import (
"io/fs"
"os"
"strings"
"github.com/datahearth/config-mapper/internal/misc"
)
type Index struct {
@ -24,7 +26,7 @@ type Indexer interface {
func NewIndexer(repoPath string) (Indexer, error) {
perms := fs.FileMode(0755)
indexPath, err := absolutePath(fmt.Sprintf("%s/.index", repoPath))
indexPath, err := misc.AbsolutePath(fmt.Sprintf("%s/.index", repoPath))
if err != nil {
return nil, err
}

View File

@ -7,26 +7,29 @@ import (
"path"
"strings"
"github.com/datahearth/config-mapper/internal/configuration"
"github.com/datahearth/config-mapper/internal/git"
"github.com/datahearth/config-mapper/internal/misc"
"github.com/fatih/color"
"github.com/spf13/viper"
)
type Items struct {
locations []OSLocation
locations []configuration.OSLocation
storage string
repository RepositoryActions
repository git.RepositoryActions
indexer Indexer
}
type ItemsActions interface {
Action(action string)
AddItems(items []OSLocation)
AddItems(items []configuration.OSLocation)
CleanUp(removedLines []string) error
}
func NewItemsActions(items []OSLocation, storage string, repository RepositoryActions, indexer Indexer) ItemsActions {
func NewItemsActions(items []configuration.OSLocation, storage string, repository git.RepositoryActions, indexer Indexer) ItemsActions {
if items == nil {
items = []OSLocation{}
items = []configuration.OSLocation{}
}
return &Items{
@ -37,17 +40,26 @@ func NewItemsActions(items []OSLocation, storage string, repository RepositoryAc
}
}
// Action performs a "save" or "load" action on all given items.
//
// Any error is printed to STDERR and item is skipped.
//
// If the performed action is "save", it'll also write the `.index` file with all new items.
func (e *Items) Action(action string) {
color.Blue("# %s", action)
color.Blue("# %s files and folders\n", action)
newLines := []string{}
for i, l := range e.locations {
var src string
storagePath, systemPath, err := configPaths(l, e.storage)
storagePath, systemPath, err := misc.ConfigPaths(l, e.storage)
if err != nil {
PrintError("[%d] failed to resolve item paths \"%v\": %v", i, l, err)
continue
}
if storagePath == "" && systemPath == "" {
color.Blue("[%d] file doesn't have configuration path for current OS. Skipping...")
continue
}
if action == "save" {
src = systemPath
@ -73,6 +85,13 @@ func (e *Items) Action(action string) {
}
}
// saveItem saves a given item inside the configured saved location.
//
// If an error is given during the process, the function returns an empty string
// (meaning the item hasn't been saved) and prints the error in STDERR.
//
// Else, returns the relative item location from the saved location to write the index
// (E.g: /home/user/.config => .config)
func (e *Items) saveItem(src, dst string, index int) string {
if err := os.MkdirAll(path.Dir(dst), 0755); err != nil {
PrintError("[%d] failed to create directory architecture for destination path \"%s\": %v", index, path.Dir(dst), err)
@ -97,24 +116,29 @@ func (e *Items) saveItem(src, dst string, index int) string {
dstPerms = s.Mode()
}
// remove the destination if it exists. It cleans up the saved location from unused files
if err := os.RemoveAll(dst); err != nil {
PrintError("[%d] failed to truncate destination folder \"%s\": %v", index, dst, err)
}
if err := os.Mkdir(dst, dstPerms); err != nil {
if !os.IsExist(err) {
PrintError("[%d] failed to create destination folder \"%s\": %v", index, dst, err)
return ""
}
}
if err := copyFolder(src, dst, true); err != nil {
if err := misc.CopyFolder(src, dst, true); err != nil {
PrintError("[%d] failed to save folder from \"%s\" to \"%s\": %v", index, src, dst, err)
return ""
}
} else {
if err := copyFile(src, dst); err != nil {
if err := misc.CopyFile(src, dst); err != nil {
PrintError("[%d] failed to save file from \"%s\" to \"%s\": %v", index, src, dst, err)
return ""
}
}
p, err := absolutePath(e.storage)
p, err := misc.AbsolutePath(e.storage)
if err != nil {
PrintError("[%d] failed resolve absolute path from configuration storage: %v", index, err)
return ""
@ -123,6 +147,10 @@ func (e *Items) saveItem(src, dst string, index int) string {
return strings.ReplaceAll(dst, p+"/", "")
}
// loadItem loads a given item onto the system.
//
// If an error is given during the process, the function returns an empty string
// (meaning the item hasn't been saved) and prints the error in STDERR.
func (e *Items) loadItem(src, dst string, index int) {
if err := os.MkdirAll(path.Dir(dst), 0755); err != nil {
PrintError("[%d] failed to create directory architecture for destination path \"%s\": %v", index, path.Dir(dst), err)
@ -153,25 +181,25 @@ func (e *Items) loadItem(src, dst string, index int) {
return
}
}
if err := copyFolder(src, dst, false); err != nil {
if err := misc.CopyFolder(src, dst, false); err != nil {
PrintError("[%d] failed to load folder from \"%s\" to \"%s\": %v", index, src, dst, err)
return
}
} else {
if err := copyFile(src, dst); err != nil {
if err := misc.CopyFile(src, dst); err != nil {
PrintError("[%d] failed to load file from \"%s\" to \"%s\": %v", index, src, dst, err)
return
}
}
}
func (e *Items) AddItems(items []OSLocation) {
func (e *Items) AddItems(items []configuration.OSLocation) {
e.locations = append(e.locations, items...)
}
func (e *Items) CleanUp(removedLines []string) error {
for _, l := range removedLines {
path, err := absolutePath(fmt.Sprintf("%s/%s", e.storage, l))
path, err := misc.AbsolutePath(fmt.Sprintf("%s/%s", e.storage, l))
if err != nil {
return err
}
@ -183,3 +211,7 @@ func (e *Items) CleanUp(removedLines []string) error {
return nil
}
func PrintError(err string, values ...interface{}) {
color.Error.Write([]byte(color.RedString(err+"\n", values...)))
}

View File

@ -1,19 +1,18 @@
package mapper
package misc
import (
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"path"
"runtime"
"strings"
"github.com/fatih/color"
"github.com/datahearth/config-mapper/internal/configuration"
)
func absolutePath(p string) (string, error) {
func AbsolutePath(p string) (string, error) {
finalPath := p
if strings.Contains(finalPath, "~") {
h, err := os.UserHomeDir()
@ -31,7 +30,8 @@ func absolutePath(p string) (string, error) {
if strings.Contains(s, "$") {
env := os.Getenv(s)
if env == "" {
return "", ErrInvalidEnv
return "", errors.New("found invalid environment variable in path")
}
pathPart = env
}
@ -49,12 +49,12 @@ func getPaths(p string, l string) (string, string, error) {
return "", "", errors.New("path incorrectly formatted. It requires \"source:destination\"")
}
src, err := absolutePath(strings.Replace(paths[0], "$LOCATION", l, 1))
src, err := AbsolutePath(strings.Replace(paths[0], "$LOCATION", l, 1))
if err != nil {
return "", "", err
}
dst, err := absolutePath(paths[1])
dst, err := AbsolutePath(paths[1])
if err != nil {
return "", "", err
}
@ -62,7 +62,7 @@ func getPaths(p string, l string) (string, string, error) {
return src, dst, nil
}
func copyFile(src, dst string) error {
func CopyFile(src, dst string) error {
s, err := os.Stat(src)
if err != nil {
return err
@ -91,17 +91,23 @@ func copyFile(src, dst string) error {
return nil
}
func configPaths(os OSLocation, location string) (string, string, error) {
func ConfigPaths(os configuration.OSLocation, location string) (string, string, error) {
var src, dst string
var err error
switch runtime.GOOS {
case "linux":
if os.Linux == "" {
return "", "", nil
}
src, dst, err = getPaths(os.Linux, location)
if err != nil {
return "", "", err
}
case "darwin":
if os.Darwin == "" {
return "", "", nil
}
src, dst, err = getPaths(os.Darwin, location)
if err != nil {
return "", "", err
@ -115,14 +121,14 @@ func configPaths(os OSLocation, location string) (string, string, error) {
var ignored map[string]bool
func copyFolder(src, dst string, checkIgnore bool) error {
func CopyFolder(src, dst string, checkIgnore bool) error {
items, err := os.ReadDir(src)
if err != nil {
return err
}
if checkIgnore {
f, err := ioutil.ReadFile(fmt.Sprintf("%s/.ignore", src))
f, err := os.ReadFile(fmt.Sprintf("%s/.ignore", src))
if err != nil && !errors.Is(err, io.EOF) {
if !errors.Is(err, os.ErrNotExist) {
return err
@ -158,21 +164,17 @@ func copyFolder(src, dst string, checkIgnore bool) error {
if err := os.MkdirAll(dstItem, info.Mode()); err != nil {
return err
}
if err := copyFolder(srcItem, dstItem, false); err != nil {
if err := CopyFolder(srcItem, dstItem, false); err != nil {
return err
}
continue
}
if err := copyFile(srcItem, dstItem); err != nil {
if err := CopyFile(srcItem, dstItem); err != nil {
return err
}
}
return nil
}
func PrintError(err string, values ...interface{}) {
color.Error.Write([]byte(color.RedString(err+"\n", values...)))
}

View File

@ -3,26 +3,28 @@ package mapper
import (
"errors"
"fmt"
"log"
"os"
"os/exec"
"strings"
"github.com/datahearth/config-mapper/internal/configuration"
"github.com/fatih/color"
"github.com/gernest/wow"
"github.com/gernest/wow/spin"
"github.com/spf13/viper"
"gopkg.in/yaml.v3"
)
var (
errLogger = log.New(os.Stderr, "", 0)
ErrFailedInstallation = errors.New("failed to install some packages. Please, checkout STDERR for more information")
ErrFailedSaving = errors.New("failed to save some packages. Please, checkout STDERR for more information")
ErrBrewNotAvailable = errors.New("homebrew is not available on your system")
ErrAptNotAvailable = errors.New("aptitude is not available on your system")
)
func LoadPkgs(c PkgManagers) error {
color.Blue("# Load folders into saved location")
// LoadPkgs triggers related functions with passed order
func LoadPkgs(c configuration.PkgManagers) error {
color.Blue("\n# Installing packages")
for _, pkg := range c.InstallationOrder {
switch pkg {
@ -42,13 +44,14 @@ func LoadPkgs(c PkgManagers) error {
return nil
}
func SavePkgs(cfg Configuration) error {
color.Blue("# Save user installed packages")
// SavePkgs triggers related functions with passed order
func SavePkgs(cfg configuration.Configuration) error {
color.Blue("# Saving user installed packages")
for _, pkg := range cfg.PackageManagers.InstallationOrder {
switch pkg {
case "homebrew":
if err := SaveBrewPkgs(cfg); err != nil {
if err := saveBrewPkgs(cfg); err != nil {
PrintError(err.Error())
return ErrFailedSaving
}
@ -60,12 +63,14 @@ func SavePkgs(cfg Configuration) error {
return nil
}
func SaveBrewPkgs(cfg Configuration) error {
// saveBrewPkgs gather user installed packages by running `brew leaves --installed-on-request`.
// It captures the output, parse it and save it into the configuration.
func saveBrewPkgs(cfg configuration.Configuration) error {
if _, err := exec.LookPath("brew"); err != nil {
return err
}
color.Blue("## Saving Homebrew packages")
color.Blue("\n## Saving Homebrew packages")
o, err := exec.Command("brew", "leaves", "--installed-on-request").Output()
if err != nil {
@ -88,6 +93,8 @@ func SaveBrewPkgs(cfg Configuration) error {
return nil
}
// installBrewPkgs installs homebrew packages by passing them to homebrew's CLI.
// STDERR and STDOUT are captured if verbose flag is passed.
func installBrewPkgs(pkgs []string) error {
if _, err := exec.LookPath("brew"); err != nil {
return ErrBrewNotAvailable
@ -100,36 +107,51 @@ func installBrewPkgs(pkgs []string) error {
cmd := exec.Command("brew", "install")
cmd.Args = append(cmd.Args, pkgs...)
color.Blue("## Installing Homebrew packages")
color.Blue("\n## Installing Homebrew packages")
spinner := wow.New(os.Stdout, spin.Get(spin.Dots3), " Running...")
v := viper.GetBool("verbose")
if v {
cmd.Stderr = os.Stderr
cmd.Stdout = os.Stdout
} else {
spinner.Start()
}
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
spinner.Stop()
PrintError("brew command failed: %v", err)
return err
}
color.Green("Packages intalled succesfully !")
if v {
// todo: find a way to clear spinner when done
spinner.Stop()
}
color.Green("\nPackages intalled succesfully !")
return nil
}
// installAptPkgs installs all provided "apt" packages by passing them to the Advanced Package Tool's CLI
func installAptPkgs(pkgs []string) error {
if _, err := exec.LookPath("apt-get"); err != nil {
if _, err := exec.LookPath("apt"); err != nil {
return ErrAptNotAvailable
}
if len(pkgs) == 0 {
fmt.Println("aptitude: nothing to do")
fmt.Println("apt: nothing to do")
return nil
}
cmd := exec.Command("sudo", "apt-get", "install")
cmd := exec.Command("sudo", "apt", "install")
cmd.Args = append(cmd.Args, pkgs...)
color.Blue("## Installing aptitude packages")
color.Blue("\n## Installing apt packages")
if err := cmd.Run(); err != nil {
PrintError("aptitude command failed: %v", err)
PrintError("apt command failed: %v", err)
return err
}

75
release.sh Executable file
View File

@ -0,0 +1,75 @@
#!/bin/bash
VERSION=v0.4.0
log() {
NTR=$'\033[0m' # * Neutral
INF=$'\033[0;34m' # * Blue (info)
WRN=$'\033[1;33m' # * Yellow (warning)
ERR=$'\033[1;31m' # * Red (error)
log_lvl=""
case $1 in
INFO)
log_lvl="${INF}$1"
;;
WARNING)
log_lvl="${WRN}$1"
;;
ERROR)
log_lvl="${ERR}$1"
;;
esac
log_lvl="${log_lvl}${NTR}"
msg="${log_lvl}\t$2"
echo -e "${msg}"
}
log "INFO" "checking required dependencies to create release"
if ! type git 1> /dev/null; then
log "ERROR" "\"git\" binary not available"
exit 1
fi
if ! type sd 1> /dev/null; then
log "ERROR" "\"sd\" binary not available"
exit 1
fi
if ! type gh 1> /dev/null; then
log "ERROR" "\"gh\" binary not available"
exit 1
fi
if ! type go 1> /dev/null; then
log "ERROR" "\"go\" binary not available"
exit 1
fi
if ! type git-chglog 1> /dev/null; then
log "ERROR" "\"git-chglog\" binary not available"
exit 1
fi
read -p "Enter a release version (vX.Y.Z): " release
log "INFO" "updating release version in files"
sd "Version: \"$VERSION\"" "Version: \"$release\"" cmd/cli.go
sd "VERSION=$VERSION" "VERSION=$release" release.sh
log "INFO" "updating changelog"
git-chglog --next-tag $release --output CHANGELOG.md
log "INFO" "commit & push changes"
git add .
git commit -m "$release"
git push
git tag -a $release -m $release
git push --tags
log "INFO" "building Linux binary"
GOOS=linux go build -o build/x86-x64_linux_config-mapper
log "INFO" "building Darwin binary"
GOOS=darwin go build -o build/x86-x64_darwin_config-mapper
log "INFO" "creating release"
gh release create $release -n $(git-chglog -t .chglog/RELEASE_CHANGELOG.tpl.md) build/x86-x64_*