Changes since v0.25:
Adrian Likins (47):
Merge branch 'master' into makkalot-facts
Merge branch 'facts-extreme' of git://github.com/makkalot/func into makkalot-facts
Merge branch 'facts-extreme' of git://github.com/makkalot/func into makkalot-facts
Merge branch 'facts-extreme' of git://github.com/makkalot/func into makkalot-facts
create a etc/version file with some version info in it.
add a get_version method to the CommandParser subclass and pass
remove unused --version support here, since it was broken and the
add /etc/func/version to setup.py
add /etc/func/version to spec file (make --version work)
add --version support to funcd
add --version support to func-transmit
add --version support to func-build-map
add FuncOptionParser() that adds a get_version method so subclasses
change to use command.FuncOptionParser so --version works
use FuncOptionParser instead of optparse.OptionParser so --version works
use FuncOptionParser so --version works
more info on release_process
Merge branch 'facts-extreme' of git://github.com/makkalot/func into makkalot-facts
Merge branch 'makkalot-facts'
update build docs
Merge branch 'makkalot-group'
Merge branch 'master' of ssh://alikins-upAAirDvJM/k7GrK9Rd4UGD2FQJk+8+***@public.gmane.org/git/func
Merge branch 'makkalot-logger'
add func-group to spec file
this test was running forever, so fix it so it doesn't.
bool object only takes "true"/"yes"/"1" etc for setting
add a config_items method.
change return codes that were previously returning a str rep
test case for new config_items method
we need to use the parent's exlude_group here
Merge branch 'makkalot-master'
change name of the decorator method, so it doesn't confuse our introspection
rev to 0.26
ugly if statements indeed. Also check to see if it's a non zero len list
add "ModuleNotFoundException" and use it when we try to invoke a method
remove func/codes.py, nothing was using it anymore.
set default log level to "INFO" instead of DEBUG
make overlapping global facts a warning
log exceptions as exception class, not info
be more specific in the logs about errors loading modules.
add ModuleNotFoundException used by module_loader in cases where modules
Try to make the hardware module handle smolt missing a little better.
add smolt dep
Make grep subcommand only call grep methods that exist.
make hardware facts module blow up nicely if smolt isnt installed.
Make facts.grep() search on the reported values of facts as well as
typo, s/=/== in hw_info test
Andreas Thienemann (1):
patches to force func back to python 2.3 compat level
Calen Pennington (1):
Exit with the return code from command.parse
David Ward (1):
Until recently, NetworkManager modified /etc/hosts and added the (real)
Gael Chamoulaud (1):
Makefile: fixed a problem during a "make install_hardest".
Greg Swift (14):
new module: disk, currently just provides df -P or df -P /partition
consolidated code so that inventory and verify rely on glob
disk.py - Adjusted return values to be strings instead of integers since XMLRPC fails on larger than 32bit numbers
remove rhel3-specific bits from func spec file
added rhel4/py2.3 sha support into getfile using same method as copyfile
Added delegate as a default Overlord config option, including placing it in the overlord.conf
Added import of yum into rpms module to allow verify to run
In virt.py module fixed a typo preventing get_maxMemory from working
Fixed help message in func-command to display \n instead of an actual new line
Added file system time to the data gathered by disk modules' usage call
To better handle internationalized systems:
Added a short cli option for --oneline to -1 (one) for func-command
adjusted glob parsing in overlord client and delegation_tools to properly "resolve" fqdn globs for systems that rely on CNAME instead of A+PTR
Added new cpu.py module to exposes cpu information. It has two methods:
Hany Fahim (1):
when we encounter a downed host, don't break, continue
John Eckersberg (8):
stray parens breaking rpms.py
base64 encode all strings before transmitting over xmlrpc
encode tracebacks
Backwards compatibility for xmlrpc Binary encoding
Use PuppetMinions for ping if configured as such
Revert "Backwards compatibility for xmlrpc Binary encoding"
Revert "encode tracebacks"
Revert "base64 encode all strings before transmitting over xmlrpc"
Louis Coilliot (1):
add getfile module to allow retrieving arbitrary files from a minion
Luca Foppiano (1):
updated jboss module
Ralph Bean (1):
added recursive parameter to filetracker.track
Ricky Zhou (2):
This allows for easier filtering out of shutdown vms.
patch from ricky zhou to fix func running under python 2.7
S.ÃaÄlar Onur (13):
Fix exceptions.NameError global name 'module_module' is not defined
Add allow_none parameter to support passing None type over XMLRPC
Solve "AttributeError: Show instance has no attribute 'conffile'" error
fix delegated async calls
commit 2fb87602b573b6e6dd21ab000da79804b3ea10f5 replaced mktemp with mkstemp but mkstemp returns a tuple containing an OS-level handle to an open file and the absolute pathname of that file, in that order. Use the latter.
certmaster returns None for minion only setups with commit f6ee54c2457bec39c869d9526d47e977e87c702c.
add dropped part of the commit c7ea92da476ffbe5bd926c84c620be0683a26e45
* Add globbing support to delegation mode,
exclude downed_hosts list in every case
start with None assigned ip variable to solve following;
Huge whitespace cleanup which includes;
modify distutils file for new commands
Exclude down hosts otherwise func "*" ping command causes following error;
Seth Vidal (70):
get rid of the newer python if convention so rpms module won't die on
capitalization fix :)
config code changes and basic puppetminions class to have func get it's
use our ca/cert/keys specified in our overlord config
have funcd use our specified ca/cert/keys and if we have turned off use_certmaster don't
add support for a -c/--conf overlord.conf conf file option to the
add listen port to overlord config so there is a default port in
rework how the overlord object and the minions object play together.
add in the option of using the openSSL.crypto.load_crl method if it is available
make sure when we get back errors connecting to list grep modules
add a host_down_list - a flat (and optional) list of hosts which are known to be down
fixup puppet minions a bit
fix wrong indentation :(
Merge branch 'master' of ssh://git.fedorahosted.org/git/func
clean up some very minor items in the yumcmd module
fix obvious indentation error.
cleanup the import sha's from func copyfile so we don't emit a deprecation
if we do not find a match in the minions list, then if the glob we're passed
move around the imports of sys and re so chasing them down isn't as annoying
- make Acls.acls a property
Gael Chamoulaud <gael.chamoulaud at gmail.com> noticed that the if statement
sort the results from the simple glob/list/inventory commands
don't output on logrotate-restart of func
sort the returns from rpms.py for invetory/glob but not for verify
- add a crl_option to the config
- only emit about downed hosts when verbose is True
wouldn't it be nice if groups worked in puppetminions too? I think so
make sure groups allow ; and ,'s for multiple entries
cache the hostlist and the revoked serial numbers for puppetminions - this makes runs involving groups not take
make sure we're calling the _get_hosts_for_spec that we're given by the minions class
- apply patch from: S.ÃaÄlar Onur <caglar-***@public.gmane.orgedu> to fix search/replace issue
merge rest of delegation fix patch from S.ÃaÄlar Onur <caglar-sXtZGUMCxtn536EW/***@public.gmane.org>
make sure we catch unresolveable hostname errors and behave accordingly
Merge branch 'master' of ssh://git.fedorahosted.org/git/func
Merge branch 'master' of ssh://git.fedorahosted.org/git/func
b/c dns names can be cased but b/c we didn't handle this to begin with by lower()ing everything
fix up typo on peerroot
make the group matching lower() all the things we pass in and all groups instantiated
lots and lots of updates to the yumcmd module:
python 2.4's XMLRPCServer doesn't have 'allow_none' which makes things break here.
the ret can be a string and not a value - so if it is a string - interpret it as success
when we setup any kind of instance of the overlord object make sure we pass
the basic outputter was making a lot of assumptions of 'command.run' and wouldn't work reliably at all
different fix for the simplexmlrpcserver differences in versions of python. This is partially from
0.27 marks
fix version in the Makefile
make sure we strip spaces off of hostnames in our group parsing
- add func/overlord/scripts.py as basis for common options in func-overlord cli scripts
- fix tab/space issues all over the place (do people still have editors that use tabs?)
fix exception catch in delegation - thanks to greg
convert func-command func-down-hosts to using the common parser
Merge branch 'master' of ssh://git.fedorahosted.org/git/func
fix users for python 2.4 and un-obsfuscate this code
add --onetime and --returncodes options to func-command
Merge branch 'master' of ssh://git.fedorahosted.org/git/func
add a confirm-on-change method to func-yum and list out what hosts
add --output-to-dir option to func-command
Merge branch 'master' of ssh://git.fedorahosted.org/git/func
perms for our db files should be 0600 not 0644
misc fixes to make minion to minion make sense
- allow modules to be controlled via config option so you can list only the ones you want
allow dumping a lot more info from the funcd - makes debugging simpler
add dump_config cmd module to dump out the config of the func client
fix acls --info output and change ordering of a few items in the output
make sure we use proper tempdirs and work correctly for
Merge branch 'master' of ssh://git.fedorahosted.org/git/func
make sure ourdbs are always 600
don't output hostnames/results from empty updates data from func-yum list updates
one more place where our db-creation perms were wrong
bump to 0.28
Todd Zullinger (2):
typo fix
typo/thinko fix
Toshio Kuratomi (3):
Fix for traceback in logger
Fix for potential infinite recursion
Remove an inner loop in system.inventory()
makkalot (166):
facts module created and some basic query options added, we use a tree like logical node structure for OR and AND operations, source is easy to follow without complexities
some beautiful tests for facts print and traversing result
adding facts to the setup file ready for instalation
adding the directory so we can install via rpm
adding the magical filter exclude chaininng methods and fixing some typos we found with unittests
adding tests for funcquery
split the funcquery so it does only the basic chaining part ,pulling the data should be done in its children for best reusability
add some more magical methods to query
overlord side of the facts it justs proxies the Overlord so we can use almost the same way as we do now, and that is good because we dont touch Overlord's code for now :)
test for overlord_facts just a few for now will add more
That is te only part that is changed for facts_overlord, if we are using facts we just add an argument more which will be taken from facts_minion,easy huh
as it is claimed the new proxyquery should be working the same way as Overlord works,the number of test is increased but ..
adding the minion facts modules should do more testign here
adding the facts loader and structure for proxy stuff that wiil handle the query requests
test the loader will fill with more
update setup.py with new stuff
update the sppec file with new stuff
when serializing the queryies i forgot to put negated stuff also so that should fix it
fix a typo
imports should occur that way
caution : having heavy recursdeseriliazation for facts and also
test cases for minion_query
test cases for overlord_query more
adding keywords for func_query
adding tests for func keywords
typo in facts hardware module
activating the facts pullinmg part for FuncLogic
adding the part that actually examines the logic structure of query and gets back the final result to be sent to the overlord and also a few fixes for query keywords
fix the keyword tests most of them were broken and also adding tests for real fact method action
injecting the fact results into minion server
wee need to make a few little injections to async code
a little fix for cases when we have fatcs in our wuery and want also to call some overlord queries like job_status
async tests for overlord_proxy will add more like test_client.py
description and version keywords for fact module
more descriptive keywords added to hardware module
adding some methods that will make func query overlord part to act as same as overlord instance
i forgoto about methods without args
remove debug messages
remove debug messages
more tests for overlord query will add more in the future
new module for showing some facts info and running some facts directly
adding tests for minion facts module
add more facts to hardware module for better testing and demonstration
generic tests for facts modules so when users add new facts they wont need to write new tests for every new module ,current tests test only the string and int returning variables so if add new types in future change that code ...
making it to be a little bit like Overlord and fixing a small bug
change tests so they look same as overlord ones
remove the hardcoded localhost
remove the hardcoded localhost
move the query stuff into Overlord object so we can have better and easier access to all
update the tests so trhey can use Overlord instead of Proxy object
adding a new filter method for ORS
add an extra argument for checking if someone registered a tag with the same name before
well we öay not need all these engli
making the conflict message more undertandable again :)
adding new filtering methods specially useful for heavy chaining
remove debug messages
adding facts filter and filter_or to commandline options
with that fix we can use the same Overlord reference when chaininng in code,without worrying about overriding the previoous query in Overlord instance
remove verbose strings ...
didnt think about all possiblities here
filter incoming delegation data
we need sometimes a method that filters those active records
a simple fix that was doing some nasty stuff when a local module swas called,and also a filter for job_status return code
some typos cleaned
documentation update of facts framework
initial groups backends created (nothing tested yet just a push)
modifications to global groups class to use pluggable backends
additions so we can support --exclude in minion naming
a simple method to get hosts for a spec will be useful groups api
with new groups api we will be able to get sub groups on the fly so we dont need subgroups thing
additino for excluding
playing with configuration backend
global groups api changes nothing is tested yet so probably will have lots of mess to fix
adding some query methods to conf backend and some documentation
more extensive querying for sqlitebackend
finishing the public api of groups now need some beautiful unittests
an util method to help us to choose the right backend
typo i'm afraid that will have lots of these :)
adding to global part backend choosing part
add new groups api module
typo
print current structure
syntax errors :|
New tests for new groups api
typos and simple logical mistakes unittests started :)
lttle api change
it is better all of methods to obey api rules
more tests
more group tests
change in api again :)
separating the minion class a little bit so we can use it from groups
typo in conf backend
some optimizations in sqlite backend but no big change,maybe should use raw sql
syntax changes little ...
tests for adding hosts via glob
a little bit optimization but we need more
sqlite backend fixes most of typos
simple mistakes in group pulling
tests for group pulling
conf backend fixes typos mostly
a parameter was forgotten :)
tests for get_hosts and get_hosts_glob methods
typos found via tests
little adjustements in groups public api
finishing the tests for groups api ,now goto cli implementation
a typo
make the default backend to be conf it is better for smaller projects
a fix for getting hosts in conf backend
a fix in set unify code and some list all behavior adjustments
starter for creating dummy minions for testing purposes
the cli code for groups
sometimes we have strange behaviour with those defaults
changing and removing some defaults
when testing it is important to have fresh copies of groups instances like we do in groups cli
some exclude option fixes in cli code
using the overlord.conf instead of certmaster.conf
adding the new goroup dir in spec
a few fixes in minion lookup code i discovered during unittests
remove the old group tests
tests for new minion api with exclude keyword inserted
more tests for minion lookup
adding exclude keyword to the Overlord
adding exclude keyword to the call cli
removing the old test files for groups api
some scalability tests for minion/cert lookup
removing the --exclude parameter from call subcommand to parent command
adding a new executable group cli insetad of using the func * thing
because the current impementation was removing hosts from globs that are in our system,it may not be possible to remove some old machine which doesnt exists anymore so that fixes it,i'm not sure if should do the same thing for _add_host
alikin's suggestion for @group:host1,host2 thing
find module added as a grep utility to func modules
some more ignores
find module for hardware
a little fix in copy of the overlord we should copy the config objects
Merge branch 'master' into modules
changing name from find to grep and implementing for modules that are in the system will add more of course
i'm going to use pydev for a little to see if it is better than vim :)
a little fix for methods without args were giving error because of facts
Merge branch 'master' into modules
more grep methods added to moduels
it seems that we dont need that one not very reasonable
the grep subcommand added so we can use it from cli
a directory for logging minions per method call or job_id
adding a new arg to args that come from overlord which is minion job_id,it will be popped before method is called we need that hacky thing to be able to log methods per their calls in different files
new loggers which are not signletons because they will write on different log files per calls
a wrapper decorator which sets a logger attached with job_id as file destination to called method and a metaclass magic which applies that wrapper to all defined module methods around that subclass the main func API class
we create always a fresh copy of each methods because we set em some logger instances and dont want they to share ssame instances which will make them to log on the same place,not very effective
some useful util methods
some test method to test logging capabilites
a clientside method which will get the tailed output
overlord side tail poller
an util db
addign part that matches overlrod-id --> minion-id
little mistake here
more controls on tail log and will add more vey soon :)
a few justifications in log polling ...
adding log polling into cmd modules with --logpoll parameter,i'm not sure actually how useful can be for 100 minions or more :)
make them to be importable
adding ability to tail_log to log one host and remove old job ids
log for one host via cmd
adding some workarounds in rferesh instance method
with new stuff the get_args methods were broken so we get em fixed that way a hacky solution ...
adding progress logger on minion side,so we can poll for progress report
adding progress poller on Overlord
making a method to log its progress
adding a progress bar that uses curses so other cli apps can use it very handy
adding a progress poller to show progres bar while polling minions
---
.gitignore | 3
Makefile | 15
README | 2
contrib/find_resources.py | 8
contrib/packagekit.py | 6
docs/release_process.txt | 25
etc/Test.conf | 2
etc/func_rotate | 2
etc/minion.conf | 5
etc/overlord.conf | 3
examples/drive_checker.py | 3
examples/find_recalled_parts.py | 4
examples/service_checker.py | 2
func.spec | 62
func/CommonErrors.py | 1
func/codes.py | 26
func/commonconfig.py | 24
func/forkbomb.py | 23
func/index_db.py | 143 +
func/jobthing.py | 113
func/logger.py | 98
func/minion/AuthedXMLRPCServer.py | 5
func/minion/acls.py | 74
func/minion/codes.py | 6
func/minion/facts/minion_query.py | 422 +++
func/minion/facts/modules/fact_module.py | 59
func/minion/facts/modules/hardware.py | 69
func/minion/facts/overlord_query.py | 99
func/minion/facts/query.py | 277 ++
func/minion/facts/query_utils.py | 40
func/minion/facts/tree.py | 119
func/minion/func_arg.py | 46
func/minion/modules/bridge.py | 15
func/minion/modules/certmastermod.py | 8
func/minion/modules/command.py | 1
func/minion/modules/confmgt_augeas.py | 23
func/minion/modules/copyfile.py | 19
func/minion/modules/cpu.py | 118
func/minion/modules/delegation.py | 22
func/minion/modules/disk.py | 61
func/minion/modules/echo.py | 29
func/minion/modules/fact.py | 88
func/minion/modules/filetracker.py | 58
func/minion/modules/func_getargs.py | 15
func/minion/modules/func_module.py | 95
func/minion/modules/getfile.py | 75
func/minion/modules/hardware.py | 196 -
func/minion/modules/httpd.py | 2
func/minion/modules/iptables/common.py | 2
func/minion/modules/iptables/port.py | 18
func/minion/modules/jboss.py | 205 -
func/minion/modules/jobs.py | 117
func/minion/modules/mount.py | 23
func/minion/modules/nagios-check.py | 2
func/minion/modules/netapp/common.py | 3
func/minion/modules/netapp/options.py | 3
func/minion/modules/netapp/snap.py | 5
func/minion/modules/netapp/vol/__init__.py | 10
func/minion/modules/netapp/vol/clone.py | 10
func/minion/modules/networktest.py | 25
func/minion/modules/overlord.py | 4
func/minion/modules/process.py | 48
func/minion/modules/pullfile.py | 1
func/minion/modules/reboot.py | 1
func/minion/modules/rpms.py | 77
func/minion/modules/service.py | 78
func/minion/modules/smart.py | 14
func/minion/modules/snmp.py | 4
func/minion/modules/sysctl.py | 17
func/minion/modules/test.py | 21
func/minion/modules/users.py | 3288 ++++++++++++------------
func/minion/modules/virt.py | 244 +
func/minion/modules/vlan.py | 8
func/minion/modules/yumcmd.py | 157 +
func/minion/server.py | 175 +
func/module_loader.py | 22
func/overlord/base_command.py | 9
func/overlord/client.py | 742 ++++-
func/overlord/cmd_modules/call.py | 309 ++
func/overlord/cmd_modules/check.py | 86
func/overlord/cmd_modules/copyfile.py | 2
func/overlord/cmd_modules/dumpconfig.py | 46
func/overlord/cmd_modules/grep.py | 152 +
func/overlord/cmd_modules/group.py | 227 +
func/overlord/cmd_modules/listminions.py | 7
func/overlord/cmd_modules/ping.py | 4
func/overlord/cmd_modules/show.py | 22
func/overlord/cmd_modules/show_hardware.py | 14
func/overlord/command.py | 19
func/overlord/delegation_tools.py | 73
func/overlord/func_command.py | 24
func/overlord/group/base.py | 72
func/overlord/group/conf_backend.py | 264 +
func/overlord/group/sqlite_backend.py | 243 +
func/overlord/groups.py | 463 +--
func/overlord/inventory.py | 27
func/overlord/mapper.py | 38
func/overlord/modules/copyfile.py | 4
func/overlord/modules/getfile.py | 113
func/overlord/modules/netapp.py | 4
func/overlord/modules/utils.py | 4
func/overlord/scripts.py | 52
func/overlord/sslclient.py | 8
func/utils.py | 404 ++
func/yaml/dump.py | 42
func/yaml/load.py | 2
func/yaml/ordered_dict.py | 4
func/yaml/redump.py | 1
func/yaml/stream.py | 8
func/yaml/timestamp.py | 52
func/yaml/ypath.py | 6
funcweb/funcweb/async_tools.py | 25
funcweb/funcweb/commands.py | 8
funcweb/funcweb/controllers.py | 102
funcweb/funcweb/identity/pam.py | 2
funcweb/funcweb/result_handler.py | 36
funcweb/funcweb/tests/bork.py | 2
funcweb/funcweb/tests/test_async_tools.py | 56
funcweb/funcweb/tests/test_client_rendering.py | 5
funcweb/funcweb/tests/test_model.py | 1
funcweb/funcweb/tests/test_result_handler.py | 9
funcweb/funcweb/tests/test_widget_automation.py | 57
funcweb/funcweb/tests/test_widget_validation.py | 79
funcweb/funcweb/widget_automation.py | 52
funcweb/funcweb/widget_validation.py | 74
pusher.py | 41
scripts/func | 6
scripts/func-command | 85
scripts/func-down-hosts | 36
scripts/func-find-user | 73
scripts/func-grep | 76
scripts/func-group | 11
scripts/func-list-vms-per-host | 59
scripts/func-ps-compare | 71
scripts/func-transmit | 3
scripts/func-whatsmyname | 58
scripts/func-yum | 665 ++++
setup.py | 136
test/async_test.py | 107
test/test-confmgt_augeas.py | 106
test/test_func_db.py | 36
test/unittest/cover_to_html.py | 68
test/unittest/plugins/funccover.py | 17
test/unittest/test_client.py | 106
test/unittest/test_config.py | 43
test/unittest/test_facts_minion.py | 340 ++
test/unittest/test_facts_overlord.py | 151 +
test/unittest/test_facts_tree.py | 105
test/unittest/test_func_arg.py | 27
test/unittest/test_func_transmit.py | 36
test/unittest/test_groups.py | 359 --
test/unittest/test_groups_api.py | 522 +++
test/unittest/test_minion.py | 193 +
test/unittest/test_users.py | 722 ++---
154 files changed, 11227 insertions(+), 4279 deletions(-)
---