Index: tools/tools/smoketestsuite/LICENSE =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/LICENSE @@ -0,0 +1,23 @@ +Copyright (c) 2017, Shivansh Rai +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Index: tools/tools/smoketestsuite/README.md =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/README.md @@ -0,0 +1,55 @@ +# Smoke testing of base utilities (FreeBSD) + +Test generation tool made as a part of [Google Summer of Code '17 with FreeBSD](https://summerofcode.withgoogle.com/projects/#6426676740227072). +**Refer the [FreeBSD wiki](https://wiki.freebsd.org/SummerOfCode2017/SmokeTestingOfBaseUtilities) for an overview and updates.** + +## Directory Structure +``` +. +├── deprecated_tests +│   └── ........................:: Tests pertaining to initial test plan +└── src ........................:: Automation tool pertaining to new test plan + ├── annotations + │   └── ....................:: Annotation files (generated/user-defined) + ├── generated_tests + │   └── ....................:: Generated atf-sh test scripts + ├── scripts + │ └── ....................:: Handy scripts + ├── generate_test.cpp ......:: Test generator + ├── read_annotations.cpp ...:: Annotation parser + └── utils.cpp ..............:: Index generator +``` + +## Automation tool +An implementation of the automation tool briefly described [here](https://lists.freebsd.org/pipermail/soc-status/2017-July/001079.html). +The following diagram summarizes how different components fit with the testcase-generator - + +![Automation-Tool](architecture.png) + +- - - + +## Dependencies +* Boost C++ libraries : The tool was tested to work with the port `boost-all-1.64.0`. + +## Instructions + +Clone the repository at `$HOME/smoketestsuite`. +The location `$HOME/smoketestsuite` is important! If using a different location, the scripts under [src/scripts](src/scripts) need to be updated accordingly (for the time being). + +### Populating groff scripts +* The directory [src/groff](src/groff) should be populated with the relevant groff scripts before proceeding for test generation. These scripts are available in the FreeBSD source tree. For filtering the utilities section wise, [fetch_groff.sh](src/scripts/fetch_groff.sh) sets the variable `section` to a default value of **1**. This value can be changed at will. However, it should be noted that currently the tool is tested to successfully generate **section 1 utilities** and might probably fail for other section numbers. + +* The variable `src` in [fetch_groff.sh](src/scripts/fetch_groff.sh) should be updated to the location of the local FreeBSD source. The default value is `$HOME/freebsd`. + +* For populating `src/groff`, execute the following inside [src](src) - + ``` + make fetch_utils + make fetch_groff + ``` + +### Generating tests +Execute the following inside [src](src) - +``` +make clean +make && make run +``` Index: tools/tools/smoketestsuite/src/Makefile =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/Makefile @@ -0,0 +1,43 @@ +# $FreeBSD$ +# +# Makefile for building the automation tool + +CC = c++ +CFLAGS = -I/usr/local/include -std=c++11 +LIBS = -L/usr/local/lib \ + -lboost_system \ + -lboost_filesystem \ + -lboost_thread \ + -lboost_chrono +OBJS = utils.o \ + read_annotations.o \ + generate_license.o \ + add_testcase.o \ + generate_test.o + +generate_tests: $(OBJS) + $(CC) $(CFLAGS) -o $(.TARGET) $(.ALLSRC) $(LIBS) + +$(OBJS): $(.PREFIX).cpp $(.PREFIX).h + $(CC) $(CFLAGS) -c $(.PREFIX).cpp + +.PHONY: clean \ + clean_tests \ + fetch_groff \ + fetch_utils \ + run + +clean: + rm -f $(OBJS) generate_tests scripts/utils_list + +fetch_groff: + sh scripts/fetch_groff.sh + +fetch_utils: + sh scripts/fetch_utils.sh + +run: + @echo Generating annotations... + sh scripts/generate_annot.sh + @echo Generating test files... + ./generate_tests Index: tools/tools/smoketestsuite/src/add_testcase.h =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/add_testcase.h @@ -0,0 +1,37 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +namespace add_testcase { + void add_known_testcase(std::string, std::string, std::string, \ + std::string, std::ofstream&); + + void add_unknown_testcase(std::string, std::string, std::string, \ + int, std::string&); + + void add_noargs_testcase(std::string, std::pair, \ + std::ofstream&); +} Index: tools/tools/smoketestsuite/src/add_testcase.cpp =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/add_testcase.cpp @@ -0,0 +1,169 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +#include +#include +#include "add_testcase.h" + +// Adds a test-case for an option with known usage. +void +add_testcase::add_known_testcase(std::string option, + std::string util_with_section, + std::string descr, + std::string output, + std::ofstream& test_script) +{ + std::string testcase_name; + std::string utility = util_with_section.substr(0, + util_with_section.length() - 3); + + // Add testcase name. + test_script << "atf_test_case "; + if (!option.empty()) { + testcase_name = option; + testcase_name.append("_flag"); + } + else + testcase_name = "no_arguments"; + test_script << testcase_name + "\n"; + + // Add testcase description. + test_script << testcase_name + + "_head()\n{\n\tatf_set \"descr\" "; + if (!descr.empty()) + test_script << descr; + else + test_script << "\"Verify the usage of option \'" + option + "\'\""; + test_script << "\n}\n\n"; + + // Add body of the testcase. + test_script << testcase_name + "_body()\n{" + + "\n\tatf_check -s exit:0 -o "; + + // Match the usage output if generated. + if (!output.empty()) + test_script << "inline:\"" + output + "\" "; + else + test_script << "empty "; + test_script << utility; + + if (!option.empty()) + test_script << " -" + option; + test_script << "\n}\n\n"; +} + +// Adds a test-case for an option with unknown usage. +void +add_testcase::add_unknown_testcase(std::string option, + std::string util_with_section, + std::string output, + int exitstatus, + std::string& testcase_buffer) +{ + std::string utility = util_with_section.substr(0, + util_with_section.length() - 3); + + testcase_buffer.append("\n\tatf_check -s not-exit:0 -e "); + + // Check if a usage message was produced. + if (!output.compare(0, 6, "usage:")) + testcase_buffer.append("match:\"$usage_output\" "); + else if (!output.empty()) + testcase_buffer.append("inline:\"" + output + "\" "); + else + testcase_buffer.append("empty "); + + testcase_buffer.append(utility); + + if (!option.empty()) + testcase_buffer.append(" -" + option); +} + +// Adds a test-case for usage without any arguments. +void +add_testcase::add_noargs_testcase(std::string util_with_section, + std::pair output, + std::ofstream& test_script) +{ + std::string descr; + std::string utility = util_with_section.substr(0, + util_with_section.length() - 3); + + if (output.second) { + // An error was encountered. + test_script << std::string("atf_test_case no_arguments\n") + + "no_arguments_head()\n{\n\tatf_set \"descr\" "; + if (!output.first.empty()) { + // We expect a usage message to be generated in this case. + if (!output.first.compare(0, 6, "usage:")) { + descr = "\"Verify that " + util_with_section + + " fails and generates a valid usage" + + " message when no arguments are supplied\""; + + test_script << descr + "\n}\n\nno_arguments_body()\n{" + + "\n\tatf_check -s not-exit:0 -e match:\"$usage_output\" " + + utility; + } + else { + descr = "\"Verify that " + util_with_section + + " fails and generates a valid output" + + " when no arguments are supplied\""; + + test_script << descr + "\n}\n\nno_arguments_body()\n{" + + "\n\tatf_check -s not-exit:0 -e inline:\"" + + output.first + "\" " + + utility; + } + } + + else { + descr = "\"Verify that " + util_with_section + + "fails silently when no arguments are supplied\"" ; + test_script << descr + "\n}\n\nno_arguments_body()\n{" + + "\n\tatf_check -s not-exit:0 -e empty " + + utility; + } + + test_script << "\n}\n\n"; + } + + else { + // The command ran successfully, hence we guessed + // a correct usage for the utility under test. + if (!output.first.empty()) + descr = "\"Verify that " + util_with_section + + " executes successfully and produces a valid" + + " output when invoked without any arguments\""; + else + descr = "\"Verify that " + util_with_section + + " executes successfully and silently" + + " when invoked without any arguments\""; + + add_testcase::add_known_testcase("", util_with_section, descr, + output.first, test_script); + } +} Index: tools/tools/smoketestsuite/src/annotations/date_test.annot =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/annotations/date_test.annot @@ -0,0 +1,5 @@ +R_flag +j_flag +n_flag +no_arguments +u_flag Index: tools/tools/smoketestsuite/src/generate_license.h =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/generate_license.h @@ -0,0 +1,30 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +namespace add_license { + std::string generate_license(); +} Index: tools/tools/smoketestsuite/src/generate_license.cpp =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/generate_license.cpp @@ -0,0 +1,71 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +#include +#include "generate_license.h" + +std::string +add_license::generate_license() +{ + std::string license; + std::string tester_name; + + // Get the tester's name which will be added in the license. + std::cout << "Please enter the name to be added in the license: "; + std::cin >> tester_name; + + license = + "#\n" + "# Copyright 2017 " + tester_name + "\n" + "# All rights reserved.\n" + "#\n" + "# Redistribution and use in source and binary forms, with or without\n" + "# modification, are permitted provided that the following conditions\n" + "# are met:\n" + "# 1. Redistributions of source code must retain the above copyright\n" + "# notice, this list of conditions and the following disclaimer.\n" + "# 2. Redistributions in binary form must reproduce the above copyright\n" + "# notice, this list of conditions and the following disclaimer in the\n" + "# documentation and/or other materials provided with the distribution.\n" + "#\n" + "# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND\n" + "# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n" + "# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n" + "# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE\n" + "# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n" + "# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS\n" + "# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\n" + "# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT\n" + "# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n" + "# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF\n" + "# SUCH DAMAGE.\n" + "#\n" + "# $FreeBSD$\n" + "#\n"; + + return license; +} Index: tools/tools/smoketestsuite/src/generate_test.h =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/generate_test.h @@ -0,0 +1,33 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +#include "utils.h" + +namespace generate_test { + std::pair exec(const char*); + void generate_test(std::string, std::string); +} Index: tools/tools/smoketestsuite/src/generate_test.cpp =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/generate_test.cpp @@ -0,0 +1,309 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "add_testcase.h" +#include "generate_license.h" +#include "generate_test.h" +#include "read_annotations.h" + +#define TIMEOUT 1 // threshold (seconds) for a function call to return. + +std::string license; // license file generated during runtime. + +// Executes the passed argument "cmd" in a shell +// and returns its output and the exit status. +std::pair +generate_test::exec(const char* cmd) +{ + const int bufsize = 128; + std::array buffer; + std::string usage_output; + FILE* pipe = popen(cmd, "r"); + + if (!pipe) + throw std::runtime_error("popen() failed!"); + + try { + while (!feof(pipe)) + if (std::fgets(buffer.data(), bufsize, pipe) != NULL) + usage_output += buffer.data(); + } + catch(...) { + pclose(pipe); + throw "Unable to execute the command: " + std::string(cmd); + } + + return std::make_pair + ((std::string)usage_output, WEXITSTATUS(pclose(pipe))); +} + +// Generate a test for the given utility. +void +generate_test::generate_test(std::string utility, + std::string section) +{ + std::list ident_opt_list; // List of identified option relations. + std::vector usage_messages; // Vector to store usage messages for comparison. + std::string command; // Command to be executed in shell. + std::string descr; // Testcase description. + std::string testcase_list; // List of testcases. + std::string testcase_buffer; // Buffer for (temporarily) holding testcase data. + std::string test_file; // atf-sh test name. + std::string util_with_section; // Section number appended to utility. + std::ofstream test_ofs; // Output stream for the atf-sh test. + std::pair output; // Return value type for `exec()`. + std::unordered_set annot; // Hashset of utility specific annotations. + int temp; + + // Read annotations and populate hash set "annot". + annotations::read_annotations(utility, annot); + + util_with_section = utility + '(' + section + ')'; + + utils::opt_def f_opts; + ident_opt_list = f_opts.check_opts(utility); + test_file = "generated_tests/" + utility + "_test.sh"; + + // Add license in the generated test scripts. + test_ofs.open(test_file, std::ios::out); + test_ofs << license; + + // If a known option was encountered (i.e. `ident_opt_list` is + // populated), produce a testcase to check the validity of the + // result of that option. If no known option was encountered, + // produce testcases to verify the correct (generated) usage + // message when using the supported options incorrectly. + + // Add testcases for known options. + if (!ident_opt_list.empty()) { + for (const auto &i : ident_opt_list) { + command = utility + " -" + i->value + " 2>&1 value, util_with_section, + descr, output.first, test_ofs); + } + else { + // A usage message was produced, i.e. we + // failed to guess the correct usage. + add_testcase::add_unknown_testcase(i->value, util_with_section, output.first, + output.second, testcase_buffer); + } + testcase_list.append("\tatf_add_test_case " + i->value + "_flag\n"); + } + } + + // Add testcases for the options whose usage is not known (yet). + if (!f_opts.opt_list.empty()) { + + // For the purpose of adding a "$usage_output" variable, + // we choose the option which produces one. + // TODO Avoid double executions of an option, i.e. one while + // selecting usage message and another while generating testcase. + + if (f_opts.opt_list.size() == 1) { + // Utility supports a single option, check if it produces a usage message. + command = utility + " -" + f_opts.opt_list.front() + " 2>&1 &1 &1 &1 > utility_list; + std::string test_file; // atf-sh test name. + std::string util_name; // Utility name. + const char *tests_dir = "generated_tests/"; + struct stat sb; + struct dirent *ent; + DIR *groff_dir; + char answer; // User input to determine overwriting of test files. + int flag = 0; + + // For testing (or generating tests for only selected utilities), + // the utility_list can be populated above during declaration. + if (utility_list.empty()) { + if ((groff_dir = opendir("groff"))) { + while ((ent = readdir(groff_dir))) { + if (!strncmp(ent->d_name, ".", 1)) + continue; + + util_name = ent->d_name; + utility_list.push_back(std::make_pair + (util_name.substr(0, util_name.length() - 2), + util_name.substr(util_name.length() - 1, 1))); + } + closedir(groff_dir); + } + else { + fprintf(stderr, "Could not open the directory: ./groff\nRefer to the " + "section \"Populating groff scripts\" in README!\n"); + return EXIT_FAILURE; + } + } + + // Check if the directory 'generated_tests' exists. + if (stat(tests_dir, &sb) || !S_ISDIR(sb.st_mode)) { + boost::filesystem::path dir(tests_dir); + if (boost::filesystem::create_directory(dir)) + std::cout << "Directory created: " << tests_dir << std::endl; + } + + // Generate a license to be added in the generated scripts. + license = add_license::generate_license(); + + for (const auto &util : utility_list) { + test_file = tests_dir + util.first + "_test.sh"; + + // Check if the test file already exists. In + // case it does, confirm before proceeding. + if (!flag && !stat(test_file.c_str(), &sb)) { + std::cout << "Test file(s) already exists. Overwrite? [y/n] "; + std::cin >> answer; + + switch (answer) { + case 'n': + case 'N': + fprintf(stderr, "Stopping execution\n"); + flag = 1; + break; + + default: + // TODO capture newline character + flag = 1; + break; + } + } + + // Enable line-buffering on stdout. + setlinebuf(stdout); + + std::cout << "Generating test for: " + util.first + + '('+ util.second + ')' << " ..."; + + boost::thread api_caller(generate_test::generate_test, util.first, util.second); + if (api_caller.try_join_for(boost::chrono::seconds(TIMEOUT))) { + // API call successfully returned within TIMEOUT (seconds). + std::cout << "Successful\n"; + } + else { + // API call timed out. + std::cout << "Failed!\n"; + // Remove the incomplete test file. + remove(("generated_tests/" + util.first + "_test.sh").c_str()); + } + } + + return EXIT_SUCCESS; +} Index: tools/tools/smoketestsuite/src/read_annotations.h =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/read_annotations.h @@ -0,0 +1,33 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +#include + +namespace annotations { + void read_annotations(std::string, \ + std::unordered_set&); +} Index: tools/tools/smoketestsuite/src/read_annotations.cpp =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/read_annotations.cpp @@ -0,0 +1,58 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +#include +#include +#include +#include +#include "read_annotations.h" + +// Read the annotation files and skip +// generation of the respective tests. +void +annotations::read_annotations(std::string utility, + std::unordered_set& annot) +{ + std::string line; + // TODO do this for all the annotation files + std::ifstream annot_fstream; + annot_fstream.open("annotations/" + utility + "_test.annot"); + + while (getline(annot_fstream, line)) { + // Add a unique identifier for no_arguments testcase + if (!line.compare(0, 12, "no_arguments")) + annot.insert("*"); + + // Add flag value for supported argument testcases + // Doing so we ignore the "invalid_usage" testcase + // as it is guaranteed to always succeed. + else if (!line.compare(2, 4, "flag")) + annot.insert(line.substr(0, 1)); + } + + annot_fstream.close(); +} Index: tools/tools/smoketestsuite/src/scripts/README.md =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/scripts/README.md @@ -0,0 +1,8 @@ +# Handy scripts + +Script Name | Functionality +----------- | ------------- +[fetch_groff.sh](fetch_groff.sh) | Lists location of all section 1 utilities +[fetch_utils.sh](fetch_utils.sh) | Saves all the base utilities in the src tree in **utils_list** +[generate_annot.sh](generate_annot.sh) | Populates annotation files under [annotations](../annotations) +[validate.sh](validate.sh) | Validates side-effects of newly introduced changes in the tool Index: tools/tools/smoketestsuite/src/scripts/fetch_groff.sh =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/scripts/fetch_groff.sh @@ -0,0 +1,47 @@ +#!/bin/sh +# +# Copyright 2017 Shivansh Rai +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# Script for listing location of groff scripts of utilities in section 1. + +set -e + +src="$HOME/freebsd" +dir_list="$HOME/smoketestsuite/src/scripts/utils_list" +groff_src="$HOME/smoketestsuite/src/groff" +section=1 # Section number for filtering the base utilities + +rm -rf "$groff_src" && mkdir "$groff_src" +cd "$src" + +while IFS= read -r dir_entry +do + for file in "$dir_entry"/* + do + case "$file" in + *."$section") cp "$file" "$HOME/smoketestsuite/src/groff/" + esac + done +done< "$dir_list" Index: tools/tools/smoketestsuite/src/scripts/fetch_utils.sh =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/scripts/fetch_utils.sh @@ -0,0 +1,42 @@ +#! /bin/sh +# +# Copyright 2017 Shivansh Rai +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# Script for listing all the base utilities in FreeBSD. +# The output "utils_list" of this script is required by "fetch_groff.sh". + +set -e +src="$HOME/freebsd" +dir="$HOME/smoketestsuite/src/scripts" + +fetch_utils() { + cd "$src" + find . -name Makefile | xargs grep -l 'PROG\|PROG_CXX' \ + | sed -e 's|/Makefile$||' | cut -c 3- +} + +rm -f utils_list + +(fetch_utils) >> "$dir/utils_list" Index: tools/tools/smoketestsuite/src/scripts/generate_annot.sh =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/scripts/generate_annot.sh @@ -0,0 +1,92 @@ +#!/bin/sh +# +# Copyright 2017 Shivansh Rai +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ +# +# Script for generating annotations based on generated tests. + +pwd=$(pwd) +suffix="_test" +extension=".sh" +update_required=0 + +message="Following annotation files were updated. \n" + +printf " ++--------------------------------------------------+ +| Checking if any annotation file is to be updated | ++--------------------------------------------------+\n" + +for f in "generated_tests"/* +do + annotations="" + file=$(basename "$f") + test=${file%$extension} + utility=${test%$suffix} + test_dir="/usr/tests/bin/$utility" + + ( + cd "$test_dir" || exit + report=$(kyua report) + i=2 + + while true + do + testcase=$(printf "%s" "$report" | awk 'NR=='"$i"' {print $1}') + status=$(printf "%s" "$report" | awk 'NR=='"$i"' {print $3}') + check=$(printf "%s" "$testcase" | cut -s -f1 -d":") + + if [ "$check" != "$test" ]; then + if [ "$annotations" ]; then + if [ $update_required = 0 ]; then + printf $message + update_required=1 + fi + + annotations_file="$pwd/annotations/$test.annot" + # Append only the new annotations + printf "$annotations" > "$annotations_file.temp" + [ ! -e "$annotations_file" ] && touch "$annotations_file" + comm -13 "$annotations_file" "$annotations_file.temp" >> \ + "$annotations_file" && printf "\t%s\n" "annotations/$test.annot" + rm -f "$annotations_file.temp" + fi + + break + fi + + if [ "$status" = "failed:" ]; then + testcase=${testcase#"$test:"} + annotations="$annotations$testcase\n" + fi + + i=$((i+1)) + done + ) + +done + +printf "==============================================================================\n\n" Index: tools/tools/smoketestsuite/src/utils.h =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/utils.h @@ -0,0 +1,50 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +#include +#include + +namespace utils { + // Option relation which maps option names to + // a unique identifier in their description. + typedef struct opt_rel { + char type; // Option type: (s)short/(l)long. + std::string value; // Name of the option. + std::string keyword; // The keyword which should be looked up in the + // message (if) produced when using this option. + } opt_rel; + + class opt_def { + public: + std::list opt_list; // list of all the accepted options with unknown usage. + std::unordered_map opt_map; // Map "option value" to "option definition". + std::unordered_map::iterator opt_map_iter; + + void insert_opts(); + std::list check_opts(std::string); + }; +} Index: tools/tools/smoketestsuite/src/utils.cpp =================================================================== --- /dev/null +++ tools/tools/smoketestsuite/src/utils.cpp @@ -0,0 +1,130 @@ +// +// Copyright 2017 Shivansh Rai +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// +// THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +// OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +// HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +// LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +// OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +// SUCH DAMAGE. +// +// $FreeBSD$ + +#include +#include +#include +#include "utils.h" + +// Insert a list of user-defined option definitions +// into a hashmap. These specific option definitions +// are the ones which one can be easily tested. +void +utils::opt_def::insert_opts() +{ + // Option definitions. + opt_rel h_def; // '-h' + h_def.type = 's'; + h_def.value = "h"; + h_def.keyword = "help"; + + opt_rel v_def; // '-v' + v_def.type = 's'; + v_def.value = "v"; + v_def.keyword = "version"; + + // `opt_map` contains all the options + // which can be "easily" tested. + opt_map.insert(std::make_pair + ("h", (opt_rel)h_def)); + opt_map.insert(std::make_pair + ("v", (opt_rel)v_def)); +}; + +// For the utility under test, find the supported options +// present in the hashmap generated by insert_opts() +// and return them in a form of list of option relations. +std::list +utils::opt_def::check_opts(std::string utility) +{ + std::string line; // An individual line in a man-page. + std::string opt_name; // Name of the option. + std::string opt_ident = ".It Fl"; // Identifier for an option in man page. + std::string buffer; // Option description extracted from man-page. + std::string opt_string; // Identified option names. + int opt_pos; // Starting index of the (identified) option. + int space_index; // First occurrence of space character + // in a multi-word option definition. + std::list ident_opt_list; // List of identified option relations (opt_rel's). + + // Generate the hashmap opt_map. + insert_opts(); + + // TODO: Section number cannot be hardcoded. + std::ifstream infile("groff/" + utility + ".1"); + + // Search for all the options accepted by the + // utility and collect those present in `opt_map`. + while (std::getline(infile, line)) { + if ((opt_pos = line.find(opt_ident)) != std::string::npos) { + opt_pos += opt_ident.length() + 1; // Locate the position of option name. + + if (opt_pos > line.length()) { + // This condition will trigger when a utility + // supports an empty argument, e.g. tset(issue #9) + continue; + } + + // Check for long options ; While here, also sanitize + // multi-word option definitions in a man page to properly + // extract short options from option definitions such as: + // .It Fl r Ar seconds (taken from date(1)). + if ((space_index = line.find(" ", opt_pos + 1, 1)) + != std::string::npos) + opt_name = line.substr(opt_pos, space_index - opt_pos); + else + opt_name = line.substr(opt_pos); + + // Check if the identified option matches the identifier. + // `opt_list.back()` is the previously checked option, the + // description of which is now stored in `buffer`. + if (!opt_list.empty() && + (opt_map_iter = opt_map.find(opt_list.back())) + != opt_map.end() && + buffer.find((opt_map_iter->second).keyword) != std::string::npos) { + ident_opt_list.push_back(&(opt_map_iter->second)); + + // Since the usage of the option under test + // is known, we remove it from `opt_list`. + opt_list.pop_back(); + } + + // Update the list of valid options. + opt_list.push_back(opt_name); + + // Empty the buffer for next option's description. + buffer.clear(); + } + else { + // Collect the option description until next + // valid option definition is encountered. + buffer.append(line); + } + } + + return ident_opt_list; +}