diff --git a/PKG-INFO b/PKG-INFO index dc0c808..da9c05d 100644 --- a/PKG-INFO +++ b/PKG-INFO @@ -1,114 +1,12 @@ Metadata-Version: 2.1 Name: osrframework -Version: 0.20.1 +Version: 0.20.2 Summary: OSRFramework - A set of AGPLv3+ OSINT tools developed by i3visio analysts for online research. Home-page: http://github.com/i3visio/osrframework Author: Felix Brezo and Yaiza Rubio Author-email: contacto@i3visio.com License: COPYING -Description: OSRFramework - ============ - - OSRFramework: Open Sources Research Framework - - Copyright (C) 2014-2020 F. Brezo and Y. Rubio, i3visio - - [![Version in PyPI](https://img.shields.io/pypi/v/osrframework.svg)]() - [![License](https://img.shields.io/badge/license-GNU%20Affero%20General%20Public%20License%20Version%203%20or%20Later-blue.svg)]() - - 1 - Description - --------------- - - OSRFramework is a GNU AGPLv3+ set of libraries developed by i3visio to perform Open Source Intelligence collection tasks. - They include references to a bunch of different applications related to username checking, DNS lookups, information leaks research, deep web search, regular expressions extraction and many others. - At the same time, by means of ad-hoc Maltego transforms, OSRFramework provides a way of making these queries graphically as well as several interfaces to interact with like OSRFConsole or a Web interface. - - 2 - License: GNU AGPLv3+ - ------------------------ - - This is free software, and you are welcome to redistribute it under certain conditions. - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU Affero General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . - - - For more details on this issue, check the [COPYING](COPYING) file. - - 3 - Installation - ---------------- - - Fast way to do it on any system for a user with administration privileges: - ``` - pip3 install osrframework - ``` - You can upgrade to the latest release of the framework with: - ``` - pip3 install osrframework --upgrade - ``` - This will manage all the dependencies for you and install the latest version of the framework. - - If you needed further information on how to install OSRFramework on certain systems, note that you may need to add `export PATH=$PATH:$HOME/.local/bin` to your `~/.bashrc_profile`). This has been confirmed in some distributions, including MacOS. - In any case, we recommend you yo have a look at the [INSTALL.md](doc/INSTALL.md) file where we provide additional details for these cases. - - 4 - Basic usage - --------------- - - If everything went correctly (we hope so!), it's time for trying usufy., mailfy and so on. - But we are they - ? They are installed in your path meaning that you can open a terminal anywhere and typing the name of the program (seems to be an improvement from previous installations...). Examples: - ``` - osrf --help - usufy -n i3visio febrezo yrubiosec -p twitter facebook - searchfy -q "i3visio" - mailfy -n i3visio - ``` - - Type -h or --help to get more information about which are the parameters of each application. - - The tools installed in this package include: - - - `alais_generator`. Generates candidate nicknames based on known info about the target. **Input**: information about the target. **Output**: list of possible nicknames. - - `checkfy`. Guesses possible emails based on a list of candidate nicknames and a pattern. **Input**: list of nicknames and an email pattern. **Output**. list of emails matching the pattern.. - - `domainfy`. Finds domains that currently resolve using a given word or nickname. **Input**: liat of words. **Output**: domains using that word that currently resolve. - - `mailfy`. Find more information about emails taken as a reference either a nickname (to generate a list of possible emails) or the email list. **Input**: list of nicknames or emails. **Output**: found information about the email. - - `osrf`. Shared wrapper for the rest of the applications. All commands can also be used as `osrf usufy…`, `osrf mailfy…`, etc. - - `phonefy`. Recovers information about mobile phones linked to known spam practices. **Inputs**: list of phones. **Outputs**: Phones linked to spam. - - `searchfy`. Finds profiles linked to a fullname. **Inputs**: list of phones. **Outputs**: Known profiles linked to the query. - - `usufy`. Identifies socialmedia profiles using a given nickname. **Inputs**: list of nicknames. **Outputs**: Known profiles in socialmedia using those nicknames. - - You can find the configuration files in a folder created in your user home to define the default behaviour of the applications: - ``` - # Configuration files for Linux and MacOS - ~/.config/OSRFramework/ - # Configuration files for Windows - C:\Users\\OSRFramework\ - ``` - - OSRFramework will look for the configuration settings for each application stored there. - You can add new credentials there and if something goes wrong, you can always restore the files stored in the `defaults` subfolder. - - If you are experiencing problems, you might fight relevant information in the (FAQ Section)[doc/FAQ.md]. - - 5 - HACKING - ----------- - - If you want to extend the functionalities of OSRFramework and you do not know where to start from, check the [HACKING.md](doc/HACKING.md) file. - - 6 - AUTHORS - ----------- - - More details about the authors in the [AUTHORS.md](AUTHORS.md) file. - +Description: UNKNOWN Keywords: python osint harvesting profiling username socialmedia forums Platform: UNKNOWN Classifier: Development Status :: 4 - Beta diff --git a/README.md b/README.md index 1980f82..e8a70b8 100644 --- a/README.md +++ b/README.md @@ -41,11 +41,11 @@ Fast way to do it on any system for a user with administration privileges: ``` -pip3 install osrframework +sudo pip3 install osrframework ``` You can upgrade to the latest release of the framework with: ``` -pip3 install osrframework --upgrade +sudo pip3 install osrframework --upgrade ``` This will manage all the dependencies for you and install the latest version of the framework. @@ -56,8 +56,8 @@ --------------- If everything went correctly (we hope so!), it's time for trying usufy., mailfy and so on. -But we are they -? They are installed in your path meaning that you can open a terminal anywhere and typing the name of the program (seems to be an improvement from previous installations...). Examples: +But where are they locally? They are installed in your path meaning that you can open a terminal anywhere and typing the name of the program (seems to be an improvement from previous installations...). Examples: + ``` osrf --help usufy -n i3visio febrezo yrubiosec -p twitter facebook diff --git a/config/plugins/wrapper.py.sample b/config/plugins/wrapper.py.sample index 049621c..9ebcf03 100644 --- a/config/plugins/wrapper.py.sample +++ b/config/plugins/wrapper.py.sample @@ -1,6 +1,9 @@ -################################################################################ +# !/usr/bin/python +# -*- coding: utf-8 -*- # -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio +################################################################################## +# +# Copyright 2016-2017 Félix Brezo and Yaiza Rubio (i3visio, contacto@i3visio.com) # # This program is part of OSRFramework. You can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by @@ -15,7 +18,7 @@ # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see . # -################################################################################ +################################################################################## __author__ = "John Doe " __version__ = "1.0" @@ -24,6 +27,7 @@ import json import re import sys +import urllib2 import osrframework.utils.browser as browser from osrframework.utils.platforms import Platform diff --git a/config/plugins/wrapper_v2.py.sample b/config/plugins/wrapper_v2.py.sample index c6247fc..fe4b816 100644 --- a/config/plugins/wrapper_v2.py.sample +++ b/config/plugins/wrapper_v2.py.sample @@ -1,6 +1,8 @@ +# -*- coding: utf-8 -*- +# ################################################################################ # -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio +# Copyright 2018 Félix Brezo and Yaiza Rubio (i3visio, contacto@i3visio.com) # # This program is part of OSRFramework. You can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by @@ -21,7 +23,7 @@ __version__ = "2.0" from osrframework.utils.platforms import Platform - +import urllib2 class Demo(Platform): """ @@ -76,4 +78,4 @@ ------- Returns the collected data if exists or None if not. """ - return query if query urllib3.open("https://demo.demo/{}".format(query)) else None + return query if query urllib2.open("https://demo.demo/{}".format(query)) else None diff --git a/doc/INSTALL.md b/doc/INSTALL.md index 434ad7a..c82934d 100644 --- a/doc/INSTALL.md +++ b/doc/INSTALL.md @@ -10,20 +10,11 @@ ``` python --version ``` - If you get errors at this point or the Python version is not appeating, your system is not yet prepared. You will need to install Python 3.6 from the official website: - ``` https://www.python.org/downloads/ ``` - Follow the installation steps for your system. Note that in one step of the Windows installation process you WILL NEED to manually add c:\Python36 and C:\Python36\scripts to the system. Try again after completing this task. - -If using OSRFramework < 0.20.0 (Python 2) only, it is recommemded to uninstall the Python 2 package. - -``` -pip2 uninstall osrframework -``` Now it's the turn of the package manager. Check that you have the pip version installed. `pip` SHOULD mostly work, but just in case and for legacy compatibilities with systems also using `pip` for Python 2, `pip3` is preferred: diff --git a/osrframework/__init__.py b/osrframework/__init__.py index 4db2487..5ee7c81 100644 --- a/osrframework/__init__.py +++ b/osrframework/__init__.py @@ -17,4 +17,4 @@ # ################################################################################ -__version__ = "0.20.1" +__version__ = "0.20.2" diff --git a/osrframework/api/twitter_api.py b/osrframework/api/twitter_api.py index 84371b0..d9f59a7 100644 --- a/osrframework/api/twitter_api.py +++ b/osrframework/api/twitter_api.py @@ -1,21 +1,24 @@ -################################################################################ +#!/usr/bin/env python +# encoding: utf-8 # -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio +################################################################################## # -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by +# Copyright 2015 Félix Brezo and Yaiza Rubio (i3visio, contacto@i3visio.com) +# +# This file is part of OSRFramework. You can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. +# GNU General Public License for more details. # -# You should have received a copy of the GNU Affero General Public License +# You should have received a copy of the GNU General Public License # along with this program. If not, see . # -################################################################################ +################################################################################## import argparse import json @@ -481,7 +484,7 @@ except: # Error... We will have to wait waiting = True - print("No more queries remaining, sleeping for " + str(waitTime) +" seconds...") + print "No more queries remaining, sleeping for " + str(waitTime) +" seconds..." time.sleep(waitTime) """ new_tweets = _getNewTweets(api, screen_name, oldest=oldest) @@ -738,6 +741,7 @@ page = 0 + # print "Getting page %s of new users..." % page+1 # Making the call to the API try: newUsers = api.search_users(query, n, page) diff --git a/osrframework/domainfy.py b/osrframework/domainfy.py index 531fdc1..7456e48 100644 --- a/osrframework/domainfy.py +++ b/osrframework/domainfy.py @@ -446,10 +446,10 @@ # Configuring the processing options group_processing = parser.add_argument_group('Processing arguments', 'Configuring the way in which mailfy will process the identified profiles.') - group_processing.add_argument('-e', '--extension', metavar='', nargs='+', choices=['csv', 'gml', 'json', 'ods', 'png', 'txt', 'xls', 'xlsx' ], required=False, default=DEFAULT_VALUES["extension"], action='store', help='output extension for the summary files. Default: xls.') - group_processing.add_argument('-o', '--output-folder', metavar='', required=False, default=DEFAULT_VALUES["output_folder"], action='store', help='output folder for the generated documents. While if the paths does not exist, usufy.py will try to create; if this argument is not provided, usufy will NOT write any down any data. Check permissions if something goes wrong.') - group_processing.add_argument('-t', '--tlds', metavar='', nargs='+', choices=["all", "none"] + list(TLD.keys()), action='store', help='list of TLD types where the nick will be looked for.', required=False, default=DEFAULT_VALUES["tlds"]) - group_processing.add_argument('-u', '--user-defined', metavar='', nargs='+', action='store', help='additional TLD that will be searched.', required=False, default=DEFAULT_VALUES["user_defined"]) + group_processing.add_argument('-e', '--extension', metavar='', nargs='+', choices=['csv', 'gml', 'json', 'ods', 'png', 'txt', 'xls', 'xlsx' ], required=False, default=DEFAULT_VALUES.get("extension", ["csv"]), action='store', help='output extension for the summary files. Default: xls.') + group_processing.add_argument('-o', '--output-folder', metavar='', required=False, default=DEFAULT_VALUES.get("output_folder", "."), action='store', help='output folder for the generated documents. While if the paths does not exist, usufy.py will try to create; if this argument is not provided, usufy will NOT write any down any data. Check permissions if something goes wrong.') + group_processing.add_argument('-t', '--tlds', metavar='', nargs='+', choices=["all", "none"] + list(TLD.keys()), action='store', help='list of TLD types where the nick will be looked for.', required=False, default=DEFAULT_VALUE.get("tlds", []) + group_processing.add_argument('-u', '--user-defined', metavar='', nargs='+', action='store', help='additional TLD that will be searched.', required=False, default=DEFAULT_VALUES.get("user_defined", [])) group_processing.add_argument('-x', '--exclude', metavar='', nargs='+', required=False, default=exclude_list, action='store', help="select the domains to be avoided. The format should include the initial '.'.") group_processing.add_argument('-F', '--file-header', metavar='', required=False, default=DEFAULT_VALUES["file_header"], action='store', help='header for the output filenames to be generated. If None was provided the following will be used: profiles..' ) group_processing.add_argument('-T', '--threads', metavar='', required=False, action='store', default=int(DEFAULT_VALUES["threads"]), type=int, help='write down the number of threads to be used (default 16). If 0, the maximum number possible will be used, which may make the system feel unstable.') diff --git a/osrframework/mailfy.py b/osrframework/mailfy.py index bf394cc..3e7743a 100644 --- a/osrframework/mailfy.py +++ b/osrframework/mailfy.py @@ -346,10 +346,9 @@ print(f"\n\t[*] {i+1}/{len(emails)} Checking '{general.title(e)}'...") for pla in platforms: # This returns a json.txt! - entities = pla.get_info(query=e, mode="mailfy") - data = json.loads(entities) - if data: - results += data + entities = pla.do_mailfy(query=e) + if entities: + results += entities return results @@ -378,13 +377,13 @@ # Configuring the processing options group_processing = parser.add_argument_group('Processing arguments', 'Configuring the way in which mailfy will process the identified profiles.') - group_processing.add_argument('-e', '--extension', metavar='', nargs='+', choices=['csv', 'gml', 'json', 'ods', 'png', 'txt', 'xls', 'xlsx' ], required=False, default=DEFAULT_VALUES["extension"], action='store', help='output extension for the summary files. Default: xls.') - group_processing.add_argument('-d', '--domains', metavar='', nargs='+', choices=['all'] + EMAIL_DOMAINS, action='store', help='list of domains where the nick will be looked for.', required=False, default=DEFAULT_VALUES["domains"]) - group_processing.add_argument('-o', '--output-folder', metavar='', required=False, default=DEFAULT_VALUES["output_folder"], action='store', help='output folder for the generated documents. While if the paths does not exist, usufy.py will try to create; if this argument is not provided, usufy will NOT write any down any data. Check permissions if something goes wrong.') + group_processing.add_argument('-e', '--extension', metavar='', nargs='+', choices=['csv', 'gml', 'json', 'ods', 'png', 'txt', 'xls', 'xlsx'], required=False, default=DEFAULT_VALUES.get("extension", ["csv"]), action='store', help='output extension for the summary files. Default: csv.') + group_processing.add_argument('-d', '--domains', metavar='', nargs='+', choices=['all'] + EMAIL_DOMAINS, action='store', help='list of domains where the nick will be looked for.', required=False, default=DEFAULT_VALUES.get("domains", [])) + group_processing.add_argument('-o', '--output-folder', metavar='', required=False, default=DEFAULT_VALUES.get("output_folder"), action='store', help='output folder for the generated documents. While if the paths does not exist, usufy.py will try to create; if this argument is not provided, usufy will NOT write any down any data. Check permissions if something goes wrong.') group_processing.add_argument('-p', '--platforms', metavar='', choices=plat_options, nargs='+', required=False, default=["all"], action='store', help='select the platforms where you want to perform the search amongst the following: {}. More than one option can be selected.'.format(str(plat_options))) group_processing.add_argument('-x', '--exclude', metavar='', choices=EMAIL_DOMAINS, nargs='+', required=False, default=exclude_list, action='store', help="select the domains to be excluded from the search.") - group_processing.add_argument('-F', '--file-header', metavar='', required=False, default=DEFAULT_VALUES["file_header"], action='store', help='Header for the output filenames to be generated. If None was provided the following will be used: profiles..' ) - group_processing.add_argument('-T', '--threads', metavar='', required=False, action='store', default = int(DEFAULT_VALUES["threads"]), type=int, help='write down the number of threads to be used (default 16). If 0, the maximum number possible will be used, which may make the system feel unstable.') + group_processing.add_argument('-F', '--file-header', metavar='', required=False, default=DEFAULT_VALUES.get("file_header", "profiles"), action='store', help='Header for the output filenames to be generated. If None was provided the following will be used: profiles..' ) + group_processing.add_argument('-T', '--threads', metavar='', required=False, action='store', default = int(DEFAULT_VALUES.get("threads", 0)), type=int, help='write down the number of threads to be used (default 16). If 0, the maximum number possible will be used, which may make the system feel unstable.') group_processing.add_argument('--quiet', required=False, action='store_true', default=False, help='tells the program not to show anything.') # About options @@ -535,7 +534,7 @@ except KeyError: # API_Key not found config_path = os.path.join(configuration.get_config_path()["appPath"], "api_keys.cfg") - print(general.warning(f"No API found for HaveIBeenPwned. Request one at and add it to '{config_path}'.")) + print("\t[*] " + general.warning("No API found for HaveIBeenPwned") + f". Request one at and add it to '{config_path}'.") except KeyboardInterrupt: print(general.warning("\tStep 3 manually skipped by the user...\n")) @@ -605,15 +604,17 @@ print(general.success(general.osrf_to_text_export(results))) now = dt.datetime.now() + """#TODO fix print(f"\n{now}\tYou can find all the information collected in the following files:") + fileHeader = os.path.join(args.output_folder, args.file_header) for ext in args.extension: # Showing the output files - print(general.emphasis("\t" + fileHeader + "." + ext)) + print(general.emphasis("\t" + fileHeader + "." + ext))""" # Showing the execution time... if not args.quiet: end_time = dt.datetime.now() - print("\n{end_time}\tFinishing execution...\n") + print(f"\n{end_time}\tFinishing execution...\n") print("Total time used:\t" + general.emphasis(str(end_time - start_time))) if not args.quiet: diff --git a/osrframework/searchengines/google.py b/osrframework/searchengines/google.py index 77d330a..76e1768 100644 --- a/osrframework/searchengines/google.py +++ b/osrframework/searchengines/google.py @@ -1,21 +1,22 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio +# !/usr/bin/python +# -*- coding: cp1252 -*- +# +################################################################################## # # This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by +# it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License # along with this program. If not, see . # -################################################################################ +################################################################################## __all__ = ['search'] @@ -32,7 +33,7 @@ else: from cookielib import LWPCookieJar from urllib import quote_plus - from urllib3 import Request, urlopen + from urllib2 import Request, urlopen from urlparse import urlparse, parse_qs # Lazy import of BeautifulSoup. @@ -69,8 +70,8 @@ @return: Web page retrieved for the given URL. @raise IOError: An exception is raised on error. - @raise urllib3.URLError: An exception is raised on error. - @raise urllib3.HTTPError: An exception is raised on error. + @raise urllib2.URLError: An exception is raised on error. + @raise urllib2.HTTPError: An exception is raised on error. """ request = Request(url) request.add_header('User-Agent', @@ -234,26 +235,27 @@ # Returns a generator that yields URLs. def processSearch(query, tld='com', lang='en', num=10, start=0, stop=50, pause=2.0, only_standard=False): - """Method that recovers the URI for a search returning an i3visio-compliant json object. - - Returns: - dict.json-like object.""" - - uriList = search(query, tld=tld, lang=lang, num=int(num), start=int(start), stop=int(stop), pause=float(pause),only_standard=only_standard) - + ''' + Method that recovers the URI for a search returning an i3visio-compliant json object. + + :return: A json-like object. + + ''' + uriList = search(query, tld=tld, lang=lang, num=int(num), start=int(start), stop=int(stop), pause=float(pause),only_standard=only_standard) + # Dictionary containing the URI objects results = [] - + # Building the objects for uri in uriList: aux = {} - aux["type"] = "com.i3visio.URI" - aux["value"] = uri + aux["type"] = "i3visio.uri" + aux["value"] = uri aux["attributes"] = [] results.append(aux) return results - + # When run as a script... if __name__ == "__main__": parser = argparse.ArgumentParser(description='A package that allows the execution of searches in Google.', prog='google.py', add_help=False) @@ -271,8 +273,9 @@ groupAbout.add_argument('--version', action='version', version='%(prog)s 0.1.0', help='shows the version of the program and exists.') args = parser.parse_args() - print("Searching..." ) - + print "Searching..." + results = processSearch(args.query, tld=args.tld, lang=args.lang, num=int(args.num), start=int(args.start), stop=int(args.stop), pause=float(args.pause),only_standard=args.only_standard) - print(json.dumps(results, indent=2)) + print json.dumps(results, indent=2) + diff --git a/osrframework/searchfy.py b/osrframework/searchfy.py index be48fe8..e7c46fa 100644 --- a/osrframework/searchfy.py +++ b/osrframework/searchfy.py @@ -77,10 +77,10 @@ # Configuring the processing options group_processing = parser.add_argument_group('Processing arguments', 'Configuring the way in which searchfy will process the identified profiles.') - group_processing.add_argument('-e', '--extension', metavar='', nargs='+', choices=['csv', 'gml', 'json', 'ods', 'png', 'txt', 'xls', 'xlsx' ], required=False, default=DEFAULT_VALUES["extension"], action='store', help='output extension for the summary files. Default: xls.') - group_processing.add_argument('-F', '--file_header', metavar='', required=False, default=DEFAULT_VALUES["file_header"], action='store', help='Header for the output filenames to be generated. If None was provided the following will be used: profiles.' ) - group_processing.add_argument('-o', '--output_folder', metavar='', required=False, default=DEFAULT_VALUES["output_folder"], action='store', help='output folder for the generated documents. While if the paths does not exist, usufy.py will try to create; if this argument is not provided, usufy will NOT write any down any data. Check permissions if something goes wrong.') - group_processing.add_argument('-p', '--platforms', metavar='', choices=listAll, nargs='+', required=False, default=DEFAULT_VALUES["platforms"] ,action='store', help='select the platforms where you want to perform the search amongst the following: ' + str(listAll) + '. More than one option can be selected.') + group_processing.add_argument('-e', '--extension', metavar='', nargs='+', choices=['csv', 'gml', 'json', 'ods', 'png', 'txt', 'xls', 'xlsx' ], required=False, default=DEFAULT_VALUES.get("extension", ["csv"]), action='store', help='output extension for the summary files. Default: xls.') + group_processing.add_argument('-F', '--file_header', metavar='', required=False, default=DEFAULT_VALUES.get("file_header", "profiles"), action='store', help='Header for the output filenames to be generated. If None was provided the following will be used: profiles.' ) + group_processing.add_argument('-o', '--output_folder', metavar='', required=False, default=DEFAULT_VALUES.get("output_folder", "."), action='store', help='output folder for the generated documents. While if the paths does not exist, usufy.py will try to create; if this argument is not provided, usufy will NOT write any down any data. Check permissions if something goes wrong.') + group_processing.add_argument('-p', '--platforms', metavar='', choices=listAll, nargs='+', required=False, default=DEFAULT_VALUES.get("platforms", []) ,action='store', help='select the platforms where you want to perform the search amongst the following: ' + str(listAll) + '. More than one option can be selected.') group_processing.add_argument('-w', '--web_browser', required=False, action='store_true', help='opening the URIs returned in the default web browser.') group_processing.add_argument('-x', '--exclude', metavar='', choices=listAll, nargs='+', required=False, default=exclude_list, action='store', help='select the platforms that you want to exclude from the processing.') @@ -172,7 +172,10 @@ end_time = dt.datetime.now() print(f"\n{end_time}\tFinishing execution...\n") print("Total time used:\t" + general.emphasis(str(end_time-start_time))) - print("Average seconds/query:\t" + general.emphasis(str((end_time-start_time).total_seconds()/len(args.platforms))) +" seconds\n") + try: + print("Average seconds/query:\t" + general.emphasis(str((end_time-start_time).total_seconds()/len(args.platforms))) +" seconds\n") + except: + pass # Urging users to place an issue on Github... print(banner.footer) diff --git a/osrframework/thirdparties/blockchain_info/getBitcoinAddressDetails.py b/osrframework/thirdparties/blockchain_info/getBitcoinAddressDetails.py index 76c628e..0b188f0 100644 --- a/osrframework/thirdparties/blockchain_info/getBitcoinAddressDetails.py +++ b/osrframework/thirdparties/blockchain_info/getBitcoinAddressDetails.py @@ -17,10 +17,11 @@ # ################################################################################ + import argparse import json import sys -import urllib3 +import urllib2 def getBitcoinAddressDetails(address=None): @@ -43,7 +44,7 @@ apiURL = "https://blockchain.info/rawaddr/" + str(address) # Accessing the HIBP API - data = urllib3.urlopen(apiURL).read() + data = urllib2.urlopen(apiURL).read() # Reading the text data onto python structures jsonData = json.loads(data) diff --git a/osrframework/thirdparties/checkIfHashIsCracked.py b/osrframework/thirdparties/checkIfHashIsCracked.py index 3e62205..ef50369 100644 --- a/osrframework/thirdparties/checkIfHashIsCracked.py +++ b/osrframework/thirdparties/checkIfHashIsCracked.py @@ -21,7 +21,6 @@ import osrframework.thirdparties.md5crack_com.checkIfHashIsCracked as md5crack_com - if __name__ == "__main__": parser = argparse.ArgumentParser(description='A library that wraps a search onto md5crack.com.', prog='checkIfHashIsCracked.py', epilog="NOTE: if not provided, the API key will be searched in the config_api_keys.py file.", add_help=False) # Adding the main options diff --git a/osrframework/thirdparties/checkIpFromAlias.py b/osrframework/thirdparties/checkIpFromAlias.py index 792d603..b711e0c 100644 --- a/osrframework/thirdparties/checkIpFromAlias.py +++ b/osrframework/thirdparties/checkIpFromAlias.py @@ -22,7 +22,6 @@ import osrframework.thirdparties.resolvethem_com.processing as resolvethem_com - if __name__ == "__main__": parser = argparse.ArgumentParser(description='A library that wraps a search onto resolvethem.com.', prog='checkIPFromAlias.py', epilog="", add_help=False) # Adding the main options diff --git a/osrframework/thirdparties/checkPhoneDetails.py b/osrframework/thirdparties/checkPhoneDetails.py index 4c30e3f..a390d3f 100644 --- a/osrframework/thirdparties/checkPhoneDetails.py +++ b/osrframework/thirdparties/checkPhoneDetails.py @@ -19,9 +19,10 @@ import argparse import json +import sys +import urllib2 import osrframework.thirdparties.infobel_com.checkPhoneDetails as infobel_com - if __name__ == "__main__": parser = argparse.ArgumentParser(description='A library that wraps a search onto infobel.com via Google.', prog='checkPhoneDetails.py', add_help=False) diff --git a/osrframework/thirdparties/getBitcoinAddressDetails.py b/osrframework/thirdparties/getBitcoinAddressDetails.py index 50128ca..13bf1dd 100644 --- a/osrframework/thirdparties/getBitcoinAddressDetails.py +++ b/osrframework/thirdparties/getBitcoinAddressDetails.py @@ -17,11 +17,11 @@ # ################################################################################ + import argparse import json import osframework.thirdparties.blockchain_info as blockchain_info - if __name__ == "__main__": parser = argparse.ArgumentParser(description='A library that wraps the search about a Bitcoin address in blockchain.info.', prog='getBitcoinAddressDetails.py', epilog="", add_help=False) diff --git a/osrframework/thirdparties/infobel_com/checkPhoneDetails.py b/osrframework/thirdparties/infobel_com/checkPhoneDetails.py index c3a4354..bee532e 100644 --- a/osrframework/thirdparties/infobel_com/checkPhoneDetails.py +++ b/osrframework/thirdparties/infobel_com/checkPhoneDetails.py @@ -20,6 +20,7 @@ import argparse import json import sys +import urllib2 import osrframework.searchengines.google as google import osrframework.thirdparties.infobel_com.processing as processing @@ -90,4 +91,4 @@ args = parser.parse_args() - print(json.dumps(checkPhoneDetails(query=args.query), indent=2)) + print json.dumps(checkPhoneDetails(query=args.query), indent=2) diff --git a/osrframework/thirdparties/infobel_com/processing.py b/osrframework/thirdparties/infobel_com/processing.py index 27dcccb..8ce089c 100644 --- a/osrframework/thirdparties/infobel_com/processing.py +++ b/osrframework/thirdparties/infobel_com/processing.py @@ -24,7 +24,6 @@ import sys import osrframework.utils.browser as browser - def extractFieldsFromResult(data): ''' diff --git a/osrframework/thirdparties/ip_api_com/checkIpDetails.py b/osrframework/thirdparties/ip_api_com/checkIpDetails.py index 88fc3b4..68310ed 100644 --- a/osrframework/thirdparties/ip_api_com/checkIpDetails.py +++ b/osrframework/thirdparties/ip_api_com/checkIpDetails.py @@ -21,11 +21,12 @@ import argparse import json import sys -import urllib3 +import urllib2 def checkIpDetails(query=None): - """Method that checks if the given hash is stored in the md5crack.com website. An example of the json received: + ''' + Method that checks if the given hash is stored in the md5crack.com website. An example of the json received: { "as": "AS8560 1\u00261 Internet AG", "city": "", @@ -43,11 +44,9 @@ "zip": "" } - Args: - query (str): Query to verify. It can be either a domain or an IPv4 address. + :param query: Query to verify. It can be either a domain or an IPv4 address. - Returns: - list. Python structure for the json received. If nothing was found, it will return an empty dictionary. + :return: Python structure for the json received. If nothing was found, it will return an empty dictionary. [ { "attributes": [ @@ -76,13 +75,13 @@ "value": "51, 9" } ] - """ + ''' try: apiURL = "http://ip-api.com/json/" + query # Accessing the ip-api.com RESTful API - data = urllib3.urlopen(apiURL).read() + data = urllib2.urlopen(apiURL).read() # Reading the text data onto python structures apiData = json.loads(data) @@ -154,11 +153,11 @@ # Appending to the list of results jsonData.append(aux) + #print json.dumps(jsonData, indent = 2) return jsonData except: # No information was found, then we return a null entity return {} - if __name__ == "__main__": parser = argparse.ArgumentParser(description='A library that wraps a search onto ip-api.com.', prog='checkIPDetails.py', epilog="", add_help=False) @@ -173,4 +172,4 @@ args = parser.parse_args() - print(json.dumps(checkIpDetails(query=args.query), indent=2)) + print json.dumps(checkIpDetails(query=args.query), indent=2) diff --git a/osrframework/thirdparties/md5crack_com/checkIfHashIsCracked.py b/osrframework/thirdparties/md5crack_com/checkIfHashIsCracked.py index 30385f6..6866ee3 100644 --- a/osrframework/thirdparties/md5crack_com/checkIfHashIsCracked.py +++ b/osrframework/thirdparties/md5crack_com/checkIfHashIsCracked.py @@ -19,14 +19,14 @@ import argparse import json +import osrframework.utils.config_api_keys as config_api_keys import sys -import urllib3 - -import osrframework.utils.config_api_keys as config_api_keys +import urllib2 def checkIfHashIsCracked(hash=None, api_key=None): - """Method that checks if the given hash is stored in the md5crack.com website. + ''' + Method that checks if the given hash is stored in the md5crack.com website. :param hash: hash to verify. :param api_key: api_key to be used in md5crack.com. If not provided, the API key will be searched in the config_api_keys.py file. @@ -38,7 +38,7 @@ "parsed": "hola", "response": "The MD5 hash was cracked." } - """ + ''' # This is for i3visio if api_key is None: #api_key = raw_input("Insert the API KEY here:\t") @@ -53,14 +53,14 @@ apiURL = "http://api.md5crack.com/crack/"+ api_key +"/" + hash # Accessing the HIBP API - data = urllib3.urlopen(apiURL).read() + data = urllib2.urlopen(apiURL).read() if "\"parsed\":null" in data: data = data.replace("\"parsed\":null", "\"parsed\":\"\"") # Reading the text data onto python structures jsonData = json.loads(data) + #print json.dumps(jsonData, indent = 2) return jsonData - if __name__ == "__main__": parser = argparse.ArgumentParser(description='A library that wraps a search onto md5crack.com.', prog='checkIfHashIsCracked.py', epilog="NOTE: if not provided, the API key will be searched in the config_api_keys.py file.", add_help=False) diff --git a/osrframework/thirdparties/md5db_net/checkIfHashIsCracked.py b/osrframework/thirdparties/md5db_net/checkIfHashIsCracked.py index 9314d4e..987169a 100644 --- a/osrframework/thirdparties/md5db_net/checkIfHashIsCracked.py +++ b/osrframework/thirdparties/md5db_net/checkIfHashIsCracked.py @@ -20,7 +20,7 @@ import argparse import json import sys -import urllib3 +import urllib2 def checkIfHashIsCracked(hash=None): @@ -38,7 +38,7 @@ try: # Getting the result of the query from MD5db.net - data = urllib3.urlopen(apiURL).read() + data = urllib2.urlopen(apiURL).read() return data except: diff --git a/osrframework/thirdparties/pipl_com/checkInfo.py b/osrframework/thirdparties/pipl_com/checkInfo.py index ef02a29..8346f6f 100644 --- a/osrframework/thirdparties/pipl_com/checkInfo.py +++ b/osrframework/thirdparties/pipl_com/checkInfo.py @@ -22,10 +22,10 @@ import json import osrframework.utils.config_api_keys as config_api_keys import sys +import urllib2 from osrframework.thirdparties.pipl_com.lib.search import SearchAPIRequest, SearchAPIError from osrframework.thirdparties.pipl_com.lib import Person, Name, Address, Job - def checkInfo(email=None, username=None, api_key=None): ''' diff --git a/osrframework/thirdparties/pipl_com/lib/search.py b/osrframework/thirdparties/pipl_com/lib/search.py index ad37aac..f03e76a 100644 --- a/osrframework/thirdparties/pipl_com/lib/search.py +++ b/osrframework/thirdparties/pipl_com/lib/search.py @@ -1,7 +1,7 @@ """Python wrapper for easily making calls to Pipl's Search API. Pipl's Search API allows you to query with the information you have about -a person (his name, address, email, phone, username and more) and in response +a person (his name, address, email, phone, username and more) and in response get all the data available on him on the web. The classes contained in this module are: @@ -14,7 +14,7 @@ """ import urllib -import urllib3 +import urllib2 import itertools import threading @@ -24,7 +24,7 @@ from osrframework.thirdparties.pipl_com.lib.utils import Serializable -# Default API key value, you can set your key globally in this variable instead +# Default API key value, you can set your key globally in this variable instead # of passing it to each request object. # >>> import osrframework.thirdparties.pipl_com.lib.search # >>> osrframework.thirdparties.pipl_com.lib.search.default_api_key = '' @@ -32,79 +32,79 @@ class SearchAPIRequest(object): - + """A request to Pipl's Search API. - + Building the request from the query parameters can be done in two ways: - - Option 1 - directly and quickly (for simple requests with only few + + Option 1 - directly and quickly (for simple requests with only few parameters): - + >>> from osrframework.thirdparties.pipl_com.lib.search import SearchAPIRequest - >>> request = SearchAPIRequest(api_key='samplekey', + >>> request = SearchAPIRequest(api_key='samplekey', email='eric@cartman.com') >>> response = request.send() - - Option 2 - using the data-model (useful for more complex queries; for - example, when there are multiple parameters of the same type - such as few phones or a few addresses or when you'd like to use - information beyond the usual identifiers such as name or email, + + Option 2 - using the data-model (useful for more complex queries; for + example, when there are multiple parameters of the same type + such as few phones or a few addresses or when you'd like to use + information beyond the usual identifiers such as name or email, information like education, job, relationships etc): - + >>> from osrframework.thirdparties.pipl_com.lib.search import SearchAPIRequest >>> from osrframework.thirdparties.pipl_com.lib import Person, Name, Address, Job >>> fields = [Name(first='Eric', last='Cartman'), Address(country='US', state='CO', city='South Park'), Address(country='US', state='NY'), Job(title='Actor')] - >>> request = SearchAPIRequest(api_key='samplekey', + >>> request = SearchAPIRequest(api_key='samplekey', person=Person(fields=fields)) >>> response = request.send() The request also supports prioritizing/filtering the type of records you - prefer to get in the response (see the append_priority_rule and + prefer to get in the response (see the append_priority_rule and add_records_filter methods). - + Sending the request and getting the response is very simple and can be done - by either making a blocking call to request.send() or by making - a non-blocking call to request.send_async(callback) which sends the request + by either making a blocking call to request.send() or by making + a non-blocking call to request.send_async(callback) which sends the request asynchronously. - + """ - + HEADERS = {'User-Agent': 'osrframework.thirdparties.pipl_com/python/%s' % osrframework.thirdparties.pipl_com.lib.__version__} BASE_URL = 'http://api.pipl.com/search/v3/json/?' # HTTPS is also supported: #BASE_URL = 'https://api.pipl.com/search/v3/json/?' - - def __init__(self, api_key=None, first_name=None, middle_name=None, - last_name=None, raw_name=None, email=None, phone=None, - username=None, country=None, state=None, city=None, + + def __init__(self, api_key=None, first_name=None, middle_name=None, + last_name=None, raw_name=None, email=None, phone=None, + username=None, country=None, state=None, city=None, raw_address=None, from_age=None, to_age=None, person=None, query_params_mode='and', exact_name=False): """Initiate a new request object with given query params. - - Each request must have at least one searchable parameter, meaning - a name (at least first and last name), email, phone or username. - Multiple query params are possible (for example querying by both email + + Each request must have at least one searchable parameter, meaning + a name (at least first and last name), email, phone or username. + Multiple query params are possible (for example querying by both email and phone of the person). - + Args: - + api_key -- str, a valid API key (use "samplekey" for experimenting). - Note that you can set a default API key - (osrframework.thirdparties.pipl_com.lib.search.default_api_key = '') instead of - passing it to each request object. + Note that you can set a default API key + (osrframework.thirdparties.pipl_com.lib.search.default_api_key = '') instead of + passing it to each request object. first_name -- unicode, minimum 2 chars. - middle_name -- unicode. + middle_name -- unicode. last_name -- unicode, minimum 2 chars. - raw_name -- unicode, an unparsed name containing at least a first name + raw_name -- unicode, an unparsed name containing at least a first name and a last name. email -- unicode. - phone -- int/long. If a unicode/str is passed instead then it'll be + phone -- int/long. If a unicode/str is passed instead then it'll be striped from all non-digit characters and converted to int. IMPORTANT: Currently only US/Canada phones can be searched by - so country code is assumed to be 1, phones with different + so country code is assumed to be 1, phones with different country codes are considered invalid and will be ignored. username -- unicode, minimum 4 chars. country -- unicode, a 2 letter country code from: @@ -118,31 +118,31 @@ to_age -- int. person -- A Person object (available at osrframework.thirdparties.pipl_com.lib.Person). The person can contain every field allowed by the data-model - (see osrframework.thirdparties.pipl_com.lib.fields) and can hold multiple fields of + (see osrframework.thirdparties.pipl_com.lib.fields) and can hold multiple fields of the same type (for example: two emails, three addresses etc.) query_params_mode -- str, one of "and"/"or" (default "and"). - Advanced parameter, use only if you care about the - value of record.query_params_match in the response + Advanced parameter, use only if you care about the + value of record.query_params_match in the response records. - Each record in the response has an attribute - "query_params_match" which indicates whether the + Each record in the response has an attribute + "query_params_match" which indicates whether the record has the all fields from the query or not. - When set to "and" all query params are required in - order to get query_params_match=True, when set to + When set to "and" all query params are required in + order to get query_params_match=True, when set to "or" it's enough that the record has at least one - of each field type (so if you search with a name - and two addresses, a record with the name and one + of each field type (so if you search with a name + and two addresses, a record with the name and one of the addresses will have query_params_match=True) exact_name -- bool (default False). - If set to True the names in the query will be matched + If set to True the names in the query will be matched "as is" without compensating for nicknames or multiple - family names. For example "Jane Brown-Smith" won't return + family names. For example "Jane Brown-Smith" won't return results for "Jane Brown" in the same way "Alexandra Pitt" won't return results for "Alex Pitt". - - Each of the arguments that should have a unicode value accepts both + + Each of the arguments that should have a unicode value accepts both unicode objects and utf8 encoded str (will be decoded automatically). - + """ if person is None: person = Person() @@ -168,25 +168,25 @@ if from_age is not None or to_age is not None: dob = DOB.from_age_range(from_age or 0, to_age or 1000) person.add_fields([dob]) - + self.api_key = api_key self.person = person self.query_params_mode = query_params_mode self.exact_name = exact_name self._filter_records_by = [] self._prioritize_records_by = [] - + @staticmethod - def _prepare_filtering_params(domain=None, category=None, + def _prepare_filtering_params(domain=None, category=None, sponsored_source=None, has_field=None, - has_fields=None, query_params_match=None, + has_fields=None, query_params_match=None, query_person_match=None, **kwargs): """Transform the params to the API format, return a list of params.""" if query_params_match not in (None, True): raise ValueError('query_params_match can only be `True`') if query_person_match not in (None, True): raise ValueError('query_person_match can only be `True`') - + params = [] if domain is not None: params.append('domain:%s' % domain) @@ -205,129 +205,129 @@ for has_field in has_fields: params.append('has_field:%s' % has_field.__name__) return params - - def add_records_filter(self, domain=None, category=None, - sponsored_source=None, has_fields=None, + + def add_records_filter(self, domain=None, category=None, + sponsored_source=None, has_fields=None, query_params_match=None, query_person_match=None): """Add a new "and" filter for the records returned in the response. - - IMPORTANT: This method can be called multiple times per request for - adding multiple "and" filters, each of these "and" filters is + + IMPORTANT: This method can be called multiple times per request for + adding multiple "and" filters, each of these "and" filters is interpreted as "or" with the other filters. For example: - + >>> from osrframework.thirdparties.pipl_com.lib.search import SearchAPIRequest >>> from osrframework.thirdparties.pipl_com.lib import Phone, Job >>> request = SearchAPIRequest('samplekey', username='eric123') >>> request.add_records_filter(domain='linkedin', has_fields=[Phone]) >>> request.add_records_filter(has_fields=[Phone, Job]) - + The above request is only for records that are: (from LinkedIn AND has a phone) OR (has a phone AND has a job). Records that don't match this rule will not come back in the response. - - Please note that in case there are too many results for the query, + + Please note that in case there are too many results for the query, adding filters to the request can significantly improve the number of useful results; when you define which records interest you, you'll get records that would have otherwise be cut-off by the limit on the number of records per query. - + Args: - + domain -- str, for example "linkedin.com", you may also use "linkedin" - but note that it'll match "linkedin.*" and "*.linkedin.*" + but note that it'll match "linkedin.*" and "*.linkedin.*" (any sub-domain and any TLD). category -- str, any one of the categories defined in osrframework.thirdparties.pipl_com.lib.source.Source.categories. - sponsored_source -- bool, True means you want just the records that - come from a sponsored source and False means you + sponsored_source -- bool, True means you want just the records that + come from a sponsored source and False means you don't want these records. - has_fields -- A list of fields classes from osrframework.thirdparties.pipl_com.lib.fields, + has_fields -- A list of fields classes from osrframework.thirdparties.pipl_com.lib.fields, records must have content in all these fields. - For example: [Name, Phone] means you only want records + For example: [Name, Phone] means you only want records that has at least one name and at least one phone. - query_params_match -- True is the only possible value and it means you - want records that match all the params you passed + query_params_match -- True is the only possible value and it means you + want records that match all the params you passed in the query. query_person_match -- True is the only possible value and it means you - want records that are the same person you - queried by (only records with - query_person_match == 1.0, see the documentation + want records that are the same person you + queried by (only records with + query_person_match == 1.0, see the documentation of record.query_person_match for more details). - + ValueError is raised in any case of an invalid parameter. - + """ params = SearchAPIRequest._prepare_filtering_params(**locals()) if params: self._filter_records_by.append(' AND '.join(params)) - - def append_priority_rule(self, domain=None, category=None, - sponsored_source=None, has_field=None, + + def append_priority_rule(self, domain=None, category=None, + sponsored_source=None, has_field=None, query_params_match=None, query_person_match=None): """Append a new priority rule for the records returned in the response. - - IMPORTANT: This method can be called multiple times per request for + + IMPORTANT: This method can be called multiple times per request for adding multiple priority rules, each call can be with only one argument - and the order of the calls matter (the first rule added is the highest + and the order of the calls matter (the first rule added is the highest priority, the second is second priority etc). For example: - + >>> from osrframework.thirdparties.pipl_com.lib.search import SearchAPIRequest >>> from osrframework.thirdparties.pipl_com.lib import Phone >>> request = SearchAPIRequest('samplekey', username='eric123') >>> request.append_priority_rule(domain='linkedin') >>> request.append_priority_rule(has_field=Phone) - - In the response to the above request records from LinkedIn will be - returned before records that aren't from LinkedIn and records with - phone will be returned before records without phone. - + + In the response to the above request records from LinkedIn will be + returned before records that aren't from LinkedIn and records with + phone will be returned before records without phone. + Please note that in case there are too many results for the query, - adding priority rules to the request does not only affect the order - of the records but can significantly improve the number of useful + adding priority rules to the request does not only affect the order + of the records but can significantly improve the number of useful results; when you define which records interest you, you'll get records that would have otherwise be cut-off by the limit on the number - of records per query. + of records per query. Args: - - domain -- str, for example "linkedin.com", "linkedin" is also possible + + domain -- str, for example "linkedin.com", "linkedin" is also possible and it'll match "linkedin.*". category -- str, any one of the categories defined in osrframework.thirdparties.pipl_com.lib.source.Source.categories. - sponsored_source -- bool, True will bring the records that - come from a sponsored source first and False + sponsored_source -- bool, True will bring the records that + come from a sponsored source first and False will bring the non-sponsored records first. has_fields -- A field class from osrframework.thirdparties.pipl_com.lib.fields. - For example: has_field=Phone means you want to give + For example: has_field=Phone means you want to give a priority to records that has at least one phone. - query_params_match -- True is the only possible value and it means you - want to give a priority to records that match all + query_params_match -- True is the only possible value and it means you + want to give a priority to records that match all the params you passed in the query. query_person_match -- True is the only possible value and it means you want to give a priority to records with higher - query_person_match (see the documentation of + query_person_match (see the documentation of record.query_person_match for more details). - + ValueError is raised in any case of an invalid parameter. - + """ params = SearchAPIRequest._prepare_filtering_params(**locals()) if len(params) > 1: raise ValueError('The function should be called with one argument') if params: self._prioritize_records_by.append(params[0]) - + def validate_query_params(self, strict=True): - """Check if the request is valid and can be sent, raise ValueError if + """Check if the request is valid and can be sent, raise ValueError if not. - - `strict` is a boolean argument that defaults to True which means an + + `strict` is a boolean argument that defaults to True which means an exception is raised on every invalid query parameter, if set to False an exception is raised only when the search request cannot be performed because required query params are missing. - + """ if not (self.api_key or default_api_key): raise ValueError('API key is missing') @@ -336,9 +336,9 @@ if not self.person.is_searchable: raise ValueError('No valid name/username/phone/email in request') if strict and self.person.unsearchable_fields: - raise ValueError('Some fields are unsearchable: %s' + raise ValueError('Some fields are unsearchable: %s' % self.person.unsearchable_fields) - + @property def url(self): """The URL of the request (str).""" @@ -351,32 +351,32 @@ 'filter_records_by': self._filter_records_by, } return SearchAPIRequest.BASE_URL + urllib.urlencode(query, doseq=True) - + def send(self, strict_validation=True): """Send the request and return the response or raise SearchAPIError. - + Calling this method blocks the program until the response is returned, - if you want the request to be sent asynchronously please refer to the - send_async method. - + if you want the request to be sent asynchronously please refer to the + send_async method. + The response is returned as a SearchAPIResponse object. - - `strict_vailidation` is a bool argument that's passed to the + + `strict_vailidation` is a bool argument that's passed to the validate_query_params method. - - Raises ValueError (raised from validate_query_params), - HttpError/URLError and SearchAPIError (when the response is returned + + Raises ValueError (raised from validate_query_params), + HttpError/URLError and SearchAPIError (when the response is returned but contains an error). - + Example: - + >>> from osrframework.thirdparties.pipl_com.lib.search import SearchAPIRequest, SearchAPIError >>> request = SearchAPIRequest('samplekey', email='eric@cartman.com') >>> try: ... response = request.send() ... except SearchAPIError as e: - ... print(e.http_status_code, e) - + ... print e.http_status_code, e + """ self.validate_query_params(strict=strict_validation) query = { @@ -387,10 +387,10 @@ 'prioritize_records_by': ','.join(self._prioritize_records_by), 'filter_records_by': self._filter_records_by, } - request = urllib3.Request(url=SearchAPIRequest.BASE_URL, data=urllib.urlencode(query, True), headers=SearchAPIRequest.HEADERS) + request = urllib2.Request(url=SearchAPIRequest.BASE_URL, data=urllib.urlencode(query, True), headers=SearchAPIRequest.HEADERS) try: - json_response = urllib3.urlopen(request).read() - except urllib3.HTTPError as e: + json_response = urllib2.urlopen(request).read() + except urllib2.HTTPError as e: json_error = e.read() if not json_error: raise e @@ -399,28 +399,28 @@ except ValueError: raise e return SearchAPIResponse.from_json(json_response) - + def send_async(self, callback, strict_validation=True): """Same as send() but in a non-blocking way. - - Use this method if you want to send the request asynchronously so your + + Use this method if you want to send the request asynchronously so your program can do other things while waiting for the response. - - `callback` is a function (or other callable) with the following + + `callback` is a function (or other callable) with the following signature: callback(response=None, error=None) - + Example: - + >>> from osrframework.thirdparties.pipl_com.lib.search import SearchAPIRequest >>> >>> def my_callback(response=None, error=None): - ... print(response or error) + ... print response or error ... >>> request = SearchAPIRequest('samplekey', email='eric@cartman.com') >>> request.send_async(my_callback) >>> do_other_things() - + """ def target(): try: @@ -432,13 +432,13 @@ class SearchAPIResponse(Serializable): - + """A response from Pipl's Search API. - + A response comprises the two things returned as a result to your query: - - - A person (osrframework.thirdparties.pipl_com.lib.containers.Person) that is the deta object - representing all the information available for the person you were + + - A person (osrframework.thirdparties.pipl_com.lib.containers.Person) that is the deta object + representing all the information available for the person you were looking for. This object will only be returned when our identity-resolution engine is convinced that the information is of the person represented by your query. @@ -448,130 +448,130 @@ that the response will not contain a person object. On the other hand, if you search by a unique identifier such as email or a combination of identifiers that only lead to one person, such as - "Eric Cartman, Age 22, From South Park, CO, US", you can expect to get + "Eric Cartman, Age 22, From South Park, CO, US", you can expect to get a response containing a single person object. - - - A list of records (osrframework.thirdparties.pipl_com.lib.containers.Record) that fully/partially - match the person from your query, if the query was for "Eric Cartman from - Colorado US" the response might also contain records of "Eric Cartman - from US" (without Colorado), if you need to differentiate between records + + - A list of records (osrframework.thirdparties.pipl_com.lib.containers.Record) that fully/partially + match the person from your query, if the query was for "Eric Cartman from + Colorado US" the response might also contain records of "Eric Cartman + from US" (without Colorado), if you need to differentiate between records with full match to the query and partial match or if you want to get a score on how likely is that record to be related to the person you are - searching please refer to the record's attributes + searching please refer to the record's attributes record.query_params_match and record.query_person_match. - - The response also contains the query as it was interpreted by Pipl. This - part is useful for verification and debugging, if some query parameters - were invalid you can see in response.query that they were ignored, you can - also see how the name/address from your query were parsed in case you + + The response also contains the query as it was interpreted by Pipl. This + part is useful for verification and debugging, if some query parameters + were invalid you can see in response.query that they were ignored, you can + also see how the name/address from your query were parsed in case you passed raw_name/raw_address in the query. - - In some cases when the query isn't focused enough and can't be matched to - a specific person, such as "John Smith from US", the response also contains - a list of suggested searches. This is a list of Record objects, each of - these is an expansion of the original query, giving additional query + + In some cases when the query isn't focused enough and can't be matched to + a specific person, such as "John Smith from US", the response also contains + a list of suggested searches. This is a list of Record objects, each of + these is an expansion of the original query, giving additional query parameters so the you can zoom in on the right person. - + """ - - def __init__(self, query=None, person=None, records=None, + + def __init__(self, query=None, person=None, records=None, suggested_searches=None, warnings_=None): """Args: - + query -- A Person object with the query as interpreted by Pipl. person -- A Person object with data about the person in the query. - records -- A list of Record objects with full/partial match to the + records -- A list of Record objects with full/partial match to the query. - suggested_searches -- A list of Record objects, each of these is an + suggested_searches -- A list of Record objects, each of these is an expansion of the original query, giving additional query parameters to zoom in on the right person. - warnings_ -- A list of unicodes. A warning is returned when the query + warnings_ -- A list of unicodes. A warning is returned when the query contains a non-critical error and the search can still run. - + """ self.query = query self.person = person self.records = records or [] self.suggested_searches = suggested_searches or [] self.warnings = warnings_ or [] - + @property def query_params_matched_records(self): """Records that match all the params in the query.""" return [rec for rec in self.records if rec.query_params_match] - + @property def query_person_matched_records(self): """Records that match the person from the query. - - Note that the meaning of "match the person from the query" means "Pipl - is convinced that these records hold data about the person you're - looking for". - Remember that when Pipl is convinced about which person you're looking - for, the response also contains a Person object. This person is - created by merging all the fields and sources of these records. - + + Note that the meaning of "match the person from the query" means "Pipl + is convinced that these records hold data about the person you're + looking for". + Remember that when Pipl is convinced about which person you're looking + for, the response also contains a Person object. This person is + created by merging all the fields and sources of these records. + """ return [rec for rec in self.records if rec.query_person_match == 1.] - + def group_records(self, key_function): - """Return a dict with the records grouped by the key returned by + """Return a dict with the records grouped by the key returned by `key_function`. - + `key_function` takes a record and returns the value from the record to group by (see examples in the group_records_by_* methods below). - + The return value is a dict, a key in this dict is a key returned by `key_function` and the value is a list of all the records with this key. - + """ sorted_records = sorted(self.records, key=key_function) grouped_records = itertools.groupby(sorted_records, key=key_function) return dict([(key, list(group)) for key, group in grouped_records]) - + def group_records_by_domain(self): """Return the records grouped by the domain they came from. - + The return value is a dict, a key in this dict is a domain and the value is a list of all the records with this domain. - + """ key_function = lambda record: record.source.domain return self.group_records(key_function) - + def group_records_by_category(self): """Return the records grouped by the category of their source. - + The return value is a dict, a key in this dict is a category and the value is a list of all the records with this category. - + """ Source.validate_categories(categories) key_function = lambda record: record.source.category return self.group_records(key_function) - + def group_records_by_query_params_match(self): """Return the records grouped by their query_params_match attribute. - + The return value is a dict, a key in this dict is a query_params_match - bool (so the keys can be just True or False) and the value is a list + bool (so the keys can be just True or False) and the value is a list of all the records with this query_params_match value. - + """ key_function = lambda record: record.query_params_match return self.group_records(key_function) - + def group_records_by_query_person_match(self): """Return the records grouped by their query_person_match attribute. - + The return value is a dict, a key in this dict is a query_person_match - float and the value is a list of all the records with this + float and the value is a list of all the records with this query_person_match value. - + """ key_function = lambda record: record.query_person_match return self.group_records(key_function) - + @staticmethod def from_dict(d): """Transform the dict to a response object and return the response.""" @@ -587,12 +587,12 @@ records = [Record.from_dict(record) for record in records] suggested_searches = d.get('suggested_searches') if suggested_searches: - suggested_searches = [Record.from_dict(record) + suggested_searches = [Record.from_dict(record) for record in suggested_searches] - return SearchAPIResponse(query=query, person=person, records=records, + return SearchAPIResponse(query=query, person=person, records=records, suggested_searches=suggested_searches, warnings_=warnings_) - + def to_dict(self): """Return a dict representation of the response.""" d = {} @@ -605,14 +605,14 @@ if self.records: d['records'] = [record.to_dict() for record in self.records] if self.suggested_searches: - d['suggested_searches'] = [record.to_dict() + d['suggested_searches'] = [record.to_dict() for record in self.suggested_searches] return d - + class SearchAPIError(APIError): - - """An exception raised when the response from the search API contains an + + """An exception raised when the response from the search API contains an error.""" - + pass diff --git a/osrframework/thirdparties/resolvethem_com/processing.py b/osrframework/thirdparties/resolvethem_com/processing.py index a31dc17..9375c98 100644 --- a/osrframework/thirdparties/resolvethem_com/processing.py +++ b/osrframework/thirdparties/resolvethem_com/processing.py @@ -21,16 +21,15 @@ import argparse import re import requests - +import urllib def checkIPFromAlias(alias=None): - """Method that checks if the given alias is currently connected to Skype and returns its IP address. + """ + Method that checks if the given alias is currently connected to Skype and returns its IP address. - Args: - alias (str): Alias to be searched. + :param alias: Alias to be searched. - Returns: - dict. Python structure for the Json received. It has the following structure: + :return: Python structure for the Json received. It has the following structure: { "type": "i3visio.ip", "value": "1.1.1.1", diff --git a/osrframework/usufy.py b/osrframework/usufy.py index 31d7e87..2b5bc31 100644 --- a/osrframework/usufy.py +++ b/osrframework/usufy.py @@ -335,7 +335,7 @@ # Selecting the platforms where performing the search groupPlatforms = parser.add_argument_group('Platform selection arguments', 'Criteria for selecting the platforms where performing the search.') - groupPlatforms.add_argument('-p', '--platforms', metavar='', choices=platOptions, nargs='+', required=False, default=DEFAULT_VALUES["platforms"], action='store', help='select the platforms where you want to perform the search amongst the following: ' + str(platOptions) + '. More than one option can be selected.') + groupPlatforms.add_argument('-p', '--platforms', metavar='', choices=platOptions, nargs='+', required=False, default=DEFAULT_VALUES.get("platforms", []), action='store', help='select the platforms where you want to perform the search amongst the following: ' + str(platOptions) + '. More than one option can be selected.') groupPlatforms.add_argument('-t', '--tags', metavar='', default = [], nargs='+', required=False, action='store', help='select the list of tags that fit the platforms in which you want to perform the search. More than one option can be selected.') groupPlatforms.add_argument('-x', '--exclude', metavar='', choices=platOptions, nargs='+', required=False, default=excludeList, action='store', help='select the platforms that you want to exclude from the processing.') @@ -345,12 +345,12 @@ group_processing.add_argument('--avoid_processing', required=False, action='store_true', default=False, help='argument to force usufy NOT to perform any processing task with the valid profiles.') group_processing.add_argument('--fuzz_config', metavar='', action='store', type=argparse.FileType('r'), help='path to the fuzzing config details. Wildcards such as the domains or the nicknames should come as: , .') group_processing.add_argument('--nonvalid', metavar='', required=False, default = '\\|<>=', action='store', help="string containing the characters considered as not valid for nicknames." ) - group_processing.add_argument('-e', '--extension', metavar='', nargs='+', choices=['csv', 'gml', 'json', 'ods', 'png', 'txt', 'xls', 'xlsx' ], required=False, default=DEFAULT_VALUES["extension"], action='store', help='output extension for the summary files. Default: xls.') + group_processing.add_argument('-e', '--extension', metavar='', nargs='+', choices=['csv', 'gml', 'json', 'ods', 'png', 'txt', 'xls', 'xlsx' ], required=False, default=DEFAULT_VALUES.get("extension", ["csv"]), action='store', help='output extension for the summary files. Default: csv.') group_processing.add_argument('-L', '--logfolder', metavar=' Protocol = p.lower()[5:] + #print p, p.lower()[5:], proxy[p] try: # Adding credentials if they exist self.proxies[ p.lower()[5:] ] = proxy[p]["username"] + ":" + proxy[p]["password"] + "@" + proxy[p]["host"] + ":" + proxy[p]["port"] diff --git a/osrframework/utils/config_api_keys.py b/osrframework/utils/config_api_keys.py index ed8c73f..0bd85c0 100644 --- a/osrframework/utils/config_api_keys.py +++ b/osrframework/utils/config_api_keys.py @@ -48,7 +48,7 @@ with open(config_path, "w") as output_file: output_file.write(cont) except Exception as e: - raise errors.ConfigurationFileNotFoundError(config_path, default_config_path); + #raise errors.ConfigurationFileNotFoundError(config_path, default_config_path); return dict_api_keys # Reading the configuration file diff --git a/osrframework/utils/credentials.py b/osrframework/utils/credentials.py index 7dd632c..27cf1da 100644 --- a/osrframework/utils/credentials.py +++ b/osrframework/utils/credentials.py @@ -57,7 +57,5 @@ creds[plat] = [c] else: creds[plat] = creds[plat].append(c) - - if len(creds) > 0: - print(f"\n\t[*] {general.info(str(len(creds)))} credentials have been loaded.") + print(f"\t[*] {general.info(str(len(creds_tuples)))} credentials have been loaded.") return creds diff --git a/osrframework/utils/platform_selection.py b/osrframework/utils/platform_selection.py index c879891..46ac7ff 100644 --- a/osrframework/utils/platform_selection.py +++ b/osrframework/utils/platform_selection.py @@ -114,7 +114,7 @@ return platform_list -def get_all_platform_names_by_tag (mode = None): +def get_all_platform_names_by_tag(mode=None): """Returns the platforms in the framework grouped by tags Args: diff --git a/osrframework/utils/platforms.py b/osrframework/utils/platforms.py index 2ed88b7..ee7871b 100644 --- a/osrframework/utils/platforms.py +++ b/osrframework/utils/platforms.py @@ -21,6 +21,7 @@ import random import re import urllib +import tempfile import osrframework.utils.browser as browser import osrframework.utils.general as general @@ -69,27 +70,32 @@ def create_url(self, word, mode="phonefy"): """Method to create the URL replacing the word in the appropriate URL + Depending on the module it returns the corresponding URL. However, + a special keyword can be provided as mode ("base") to use a different + URL instead. This is useful when the check is performed towards a + different platform (e. g., tweettunnel.com) but the URI to be shown is + expected to be the original one (e. g., twitter.com). + Args: word (str): Word to be searched. mode (str): Mode to be executed. Returns: - The URL to be queried. - """ - try: - return self.modes[mode]["url"].replace("{placeholder}", word) - except: + The URL to be queried. It returns None if no base has been provided. + """ + try: if mode == "base": if word[0] == "/": - return self.baseURL + word[1:], word + return self.baseURL + word[1:] else: return self.baseURL + word else: try: + return self.modes[mode]["url"].replace("{placeholder}", word) + except: return self.url[mode].replace("<"+mode+">", word) - except: - pass - return None + except: + return None def launch_query_for_mode(self, query=None, mode=None): """Method that launches an i3Browser to collect data @@ -120,9 +126,18 @@ else: # Accessing the resources data = i3Browser.recover_url(qURL) + + try: + if self.modes[mode]["debug"]: + with open(os.path.join(tempfile.gettempdir(), self.platformName) + ".html", "w") as file: + print(f"DEBUG mode is active for '{general.emphasis(self.platformName)}'. Data grabbed saved to: '{general.warning(file.name)}'") + file.write(data) + except: + pass + return data except KeyError: - print(general.error("[*] '{}' is not a valid mode for this wrapper ({}).".format(mode, self.__class__.__name__))) + print(general.error("\t[*] '{}' is not a valid mode for this wrapper ({}).".format(mode, self.__class__.__name__))) return None @@ -350,11 +365,11 @@ ] """ if self.check_mailfy(query, **kwargs): - expandedEntities = general.expand_entities_from_email(query) + expanded_entities = general.expand_entities_from_email(query) r = { "type": "com.i3visio.Profile", "value": self.platformName + " - " + query, - "attributes": expandedEntities + [ + "attributes": expanded_entities + [ { "type": "com.i3visio.Platform", "value": self.platformName, @@ -401,7 +416,7 @@ A list of elements to be appended. """ results = [] - print(f"[*] Launching search using the {self.__class__.__name__} module...") + print(f"\t[*] Launching search using the {self.__class__.__name__} module...") test = self.check_searchfy(query, **kwargs) if test: @@ -644,7 +659,13 @@ # Appending platform URI aux = {} aux["type"] = "com.i3visio.URI" - aux["value"] = self.create_url(word=query, mode="usufy") + + uri = self.create_url(word=query, mode="base") + if uri: + aux["value"] = uri + else: + aux["value"] = self.create_url(word=query, mode="usufy") + aux["attributes"] = [] r["attributes"].append(aux) # Appending the alias diff --git a/osrframework/wrappers/bitbacker.py b/osrframework/wrappers/bitbacker.py deleted file mode 100644 index 765e9f7..0000000 --- a/osrframework/wrappers/bitbacker.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Bitbacker(Platform): - """A object for Bitbacker""" - def __init__(self): - self.platformName = "bitbacker" - self.tags = ["cryptocurrencies", "payments"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://bitbacker.io/user/" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["

Not Found

"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/bucketlistly.py b/osrframework/wrappers/bucketlistly.py deleted file mode 100644 index a30fcda..0000000 --- a/osrframework/wrappers/bucketlistly.py +++ /dev/null @@ -1,101 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Bucketlistly(Platform): - """A object for Bucketlistly""" - def __init__(self): - self.platformName = "Bucketlistly" - self.tags = ["trips"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "http://www.bucketlistly.com/users/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = "[^0-9][^\.]+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["Sorry, the page you are looking for doesn't exist. (404) - Bucketlistly"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/canva.py b/osrframework/wrappers/canva.py deleted file mode 100644 index df5aa3d..0000000 --- a/osrframework/wrappers/canva.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Canva(Platform): - """A object for Canva""" - def __init__(self): - self.platformName = "Canva" - self.tags = ["imagery"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://www.canva.com/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["(404)"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/colourlovers.py b/osrframework/wrappers/colourlovers.py deleted file mode 100644 index 2ababa4..0000000 --- a/osrframework/wrappers/colourlovers.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Colourlovers(Platform): - """A object for Colourlovers""" - def __init__(self): - self.platformName = "Colourlovers" - self.tags = ["art"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "http://www.colourlovers.com/lover/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["

No one's home

"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/douban.py b/osrframework/wrappers/douban.py index 4d1ed46..78577d5 100644 --- a/osrframework/wrappers/douban.py +++ b/osrframework/wrappers/douban.py @@ -18,84 +18,28 @@ ################################################################################ __author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" +__version__ = "3.0" from osrframework.utils.platforms import Platform class Douban(Platform): - """A object for Douban""" + """ class""" def __init__(self): + """Constructor with parameters + + This method permits the developer to instantiate dinamically Platform + objects.""" self.platformName = "Douban" - self.tags = ["social", "opinions"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://site.douban.com/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["https://img3.doubanio.com/pics/douban_error.gif"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} + self.tags = ["social"] + self.modes = { + "usufy": { + "debug": False, + "extra_fields": {}, + "needs_credentials": False, + "not_found_text": 'https://img9.doubanio.com/pics/douban_error.gif"/>', + "query_validator": "[a-z0-9A-Z_]+", + "url": "https://site.douban.com/{placeholder}", + } + } diff --git a/osrframework/wrappers/dreamstime.py b/osrframework/wrappers/dreamstime.py deleted file mode 100644 index f963951..0000000 --- a/osrframework/wrappers/dreamstime.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Dreamstime(Platform): - """A object for Dreamstime""" - def __init__(self): - self.platformName = "Dreamstime" - self.tags = ["social"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://es.dreamstime.com/" + "" + "_info" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ['error404'] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/facebook.py b/osrframework/wrappers/facebook.py index 8c41fea..125b7ca 100644 --- a/osrframework/wrappers/facebook.py +++ b/osrframework/wrappers/facebook.py @@ -33,72 +33,27 @@ # Base URL self.baseURL = "https://facebook.com/" - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = True - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://www.facebook.com/" + "" - self.url["searchfy"] = "https://www.facebook.com/public?query=" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - self.validQuery["searchfy"] = ".+" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["https://static.xx.fbcdn.net/rsrc.php/v3/yp/r/U4B06nLMGQt.png"] - self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - self.fieldsRegExp["searchfy"] = {} - self.searchfyAliasRegexp = "(.+) \| Facebook', # Regular expresion to extract the alias + }, + "needs_credentials": False, + "not_found_text": "https://static.xx.fbcdn.net/rsrc.php/v3/yp/r/U4B06nLMGQt.png", # Text that indicates a missing profile + "query_validator": "[a-zA-Z\.0-9_\-]+", # Regular expression that the alias SHOULD match + "url": "https://www.facebook.com/{placeholder}", # Target URL where {placeholder} would be modified by the alias + }, + "searchfy": { + "debug": False, + "extra_fields": {}, + "needs_credentials": False, + "not_found_text": "We couldn't find anything for", + "query_validator": ".+", + "url": "https://www.facebook.com/public?query={placeholder}", + # Needed function to extract aliases from the website + "alias_regexp": '' + }, + # Reimplementation needed of check_mailfy + "mailfy": {}, + } diff --git a/osrframework/wrappers/freelancer.py b/osrframework/wrappers/freelancer.py index 9df0c7c..7c63aaa 100644 --- a/osrframework/wrappers/freelancer.py +++ b/osrframework/wrappers/freelancer.py @@ -18,83 +18,29 @@ ################################################################################ __author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" +__version__ = "3.0" from osrframework.utils.platforms import Platform class Freelancer(Platform): - """A object for Freelancer""" + """A object for Facebook""" def __init__(self): self.platformName = "Freelancer" self.tags = ["jobs"] - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://www.freelancer.com/u/" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["404 - Page not found"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} + self.modes = { + "usufy": { + "debug": False, + "extra_fields": {}, + "needs_credentials": False, + "not_found_text": " Looks like the page you are looking for doesn't exist", # Text that indicates a missing profile + "query_validator": "[a-zA-Z\.0-9_\-]+", # Regular expression that the alias SHOULD match + "url": "https://www.freelancer.com/u/{placeholder}", # Target URL where {placeholder} would be modified by the alias + "test": { + "valid": "james", + "invalid": "7ddf32e17a6ac5" + } + } + } diff --git a/osrframework/wrappers/gnupgkeys.py b/osrframework/wrappers/gnupgkeys.py new file mode 100644 index 0000000..737128e --- /dev/null +++ b/osrframework/wrappers/gnupgkeys.py @@ -0,0 +1,208 @@ +################################################################################ +# +# Copyright 2015-2020 Félix Brezo and Yaiza Rubio +# +# This program is part of OSRFramework. You can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . +# +################################################################################ + +__author__ = "Felix Brezo, Yaiza Rubio " +__version__ = "2.0" + + +import re + +import osrframework.utils.general as general +from osrframework.utils.platforms import Platform + + +class GnuPGKeys(Platform): + """A object for the MIT PGP public keys repository""" + def __init__(self): + self.platformName = "GnuPGKeys" + self.tags = ["mails", "cryptography"] + + # Base URL + self.baseURL = "http://keys.gnupg.net/pks/lookup?search=" + ######################## + # Defining valid modes # + ######################## + self.isValidMode = {} + self.isValidMode["mailfy"] = True + self.isValidMode["phonefy"] = False + self.isValidMode["usufy"] = False + self.isValidMode["searchfy"] = True + + ###################################### + # Search URL for the different modes # + ###################################### + # Strings with the URL for each and every mode + self.url = {} + #self.url["phonefy"] = "http://anyurl.com//phone/" + "" + #self.url["usufy"] = "https://github.com/" + "" + self.url["searchfy"] = "http://keys.gnupg.net/pks/lookup?search=" + + ###################################### + # Whether the user needs credentials # + ###################################### + self.needsCredentials = {} + #self.needsCredentials["phonefy"] = False + #self.needsCredentials["usufy"] = False + self.needsCredentials["searchfy"] = False + + ################# + # Valid queries # + ################# + # Strings that will imply that the query number is not appearing + self.validQuery = {} + # The regular expression '.+' will match any query. + #self.validQuery["phonefy"] = ".*" + #self.validQuery["usufy"] = ".*" + self.validQuery["searchfy"] = ".+" + self.validQuery["mailfy"] = ".+" + + ################### + # Not_found clues # + ################### + # Strings that will imply that the query number is not appearing + self.notFoundText = {} + #self.notFoundText["phonefy"] = [] + self.notFoundText["usufy"] = [ + "This is not the web page you are looking for" + ] + self.notFoundText["searchfy"] = [] + + ######################### + # Fields to be searched # + ######################### + self.fieldsRegExp = {} + + # Definition of regular expressions to be searched in phonefy mode + #self.fieldsRegExp["phonefy"] = {} + # Example of fields: + #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" + + # Definition of regular expressions to be searched in usufy mode + #self.fieldsRegExp["usufy"] = {} + # Example of fields: + #self.fieldsRegExp["usufy"]["i3visio.location"] = "" + # Definition of regular expressions to be searched in searchfy mode + self.fieldsRegExp["searchfy"] = {} + #self.searchfyAliasRegexp = '<([^\&]+)>' + self.searchfyEmailRegexp = ' <([^\&]+)>' + + # Example of fields: + #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" + + ################ + # Fields found # + ################ + # This attribute will be feeded when running the program. + self.foundFields = {} + + def check_mailfy(self, query, **kwargs): + """Verifying a mailfy query in this platform + + This might be redefined in any class inheriting from Platform. The only + condition is that any of this should return a dictionary as defined. + + Args: + query (str): The element to be searched. + + Returns: + String. Returns the collected data if exists or None if not. + """ + import re + import requests + + s = requests.Session() + + # Getting the first response to grab the csrf_token + resp = s.get(f"http://keys.gnupg.net/pks/lookup?search={query}") + + if resp.status_code == 200 or resp.status_code == 404: + if ' 0 keys found..' in resp.text: + return None + else: + return resp.text + else: + print(general.warning(f"\t[*] Something happened. keyserver.io returned status '{resp.status_code}' for '{query}'.")) + return None + + def do_mailfy(self, query, **kwargs): + """Verifying a mailfy query in this platform + + This might be redefined in any class inheriting from Platform. The only + condition is that any of this should return an equivalent array. + + Args: + query: The element to be searched. + + Returns: + A list of elements to be appended. A sample output format is as follows: + [ + { + "attributes": [ + { + "attributes": [], + "type": "com.i3visio.Email", + "value": "contacto@i3visio.com" + }, + { + "attributes": [], + "type": "com.i3visio.Alias", + "value": "contacto" + }, + { + "attributes": [], + "type": "com.i3visio.Domain", + "value": "i3visio.com" + }, + { + "attributes": [], + "type": "com.i3visio.Platform", + "value": "Twitter" + } + ], + "type": "com.i3visio.Profile", + "value": "Twitter - contacto@i3visio.com" + } + ] + """ + info = self.check_mailfy(query, **kwargs) + + results = [] + if info: + emails = set(re.findall(' <([^\&]+)>', info)) + + for i, email in enumerate(emails): + try: + expandedEntities = general.expand_entities_from_email(email) + r = { + "type": "com.i3visio.Profile", + "value": self.platformName + " - " + email, + "attributes": expandedEntities + [ + { + "type": "com.i3visio.Platform", + "value": self.platformName, + "attributes": [] + } + ] + } + results.append(r) + except IndexError: + # A result does not contain a @. + pass + + return results diff --git a/osrframework/wrappers/infojobs.py b/osrframework/wrappers/infojobs.py index 3915c8f..ebaac8b 100644 --- a/osrframework/wrappers/infojobs.py +++ b/osrframework/wrappers/infojobs.py @@ -60,7 +60,6 @@ # This attribute will be feeded when running the program. self.foundFields = {} - def check_mailfy(self, query, kwargs={}): """Verifying a mailfy query in this platform diff --git a/osrframework/wrappers/jamiiforums.py b/osrframework/wrappers/jamiiforums.py index de75b0c..249d193 100644 --- a/osrframework/wrappers/jamiiforums.py +++ b/osrframework/wrappers/jamiiforums.py @@ -18,88 +18,30 @@ ################################################################################ __author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" +__version__ = "3.0" from osrframework.utils.platforms import Platform class Jamiiforums(Platform): - """A object for Jamiiforums""" + """ class""" def __init__(self): + """Constructor with parameters + + This method permits the developer to instantiate dinamically Platform + objects.""" self.platformName = "Jamiiforums" - self.tags = ["opinions", "contact"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "http://jamiiforums.com/members/?username=" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = [ - ( - "The specified member cannot be found. Please enter a member's" - " entire name." - ) - ] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} + self.tags = ["opinions"] + self.modes = { + "usufy": { + "debug": False, + "extra_fields": { + "com.i3visio.Name": '>([^<]+)', + }, + "needs_credentials": False, + "not_found_text": "The specified member cannot be found. Please enter a member's entire name.", + "query_validator": "[a-z0-9A-Z_]+", + "url": "https://www.jamiiforums.com/members/?username={placeholder}", + } + } diff --git a/osrframework/wrappers/key_server.py b/osrframework/wrappers/key_server.py deleted file mode 100644 index c04b45a..0000000 --- a/osrframework/wrappers/key_server.py +++ /dev/null @@ -1,208 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -import re - -import osrframework.utils.general as general -from osrframework.utils.platforms import Platform - - -class KeyServerIO(Platform): - """A object for the MIT PGP public keys repository""" - def __init__(self): - self.platformName = "KeyServerIO" - self.tags = ["mails", "cryptography"] - - # Base URL - self.baseURL = "https://pgp.key-server.io/pks/lookup?search=" - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["mailfy"] = True - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = False - self.isValidMode["searchfy"] = True - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - #self.url["usufy"] = "https://github.com/" + "" - self.url["searchfy"] = "https://pgp.key-server.io/pks/lookup?search=" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - #self.needsCredentials["usufy"] = False - self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - #self.validQuery["usufy"] = ".*" - self.validQuery["searchfy"] = ".+" - self.validQuery["mailfy"] = ".+" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = [ - "This is not the web page you are looking for" - ] - self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - #self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - self.fieldsRegExp["searchfy"] = {} - #self.searchfyAliasRegexp = '<([^\&]+)>' - self.searchfyEmailRegexp = ' <([^\&]+)>' - - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} - - def check_mailfy(self, query, **kwargs): - """Verifying a mailfy query in this platform - - This might be redefined in any class inheriting from Platform. The only - condition is that any of this should return a dictionary as defined. - - Args: - query (str): The element to be searched. - - Returns: - String. Returns the collected data if exists or None if not. - """ - import re - import requests - - s = requests.Session() - - # Getting the first response to grab the csrf_token - resp = s.get(f"https://pgp.key-server.io/pks/lookup?search={query}") - - if resp.status_code == 200 or resp.status_code == 404: - if ' 0 keys found..' in resp.text: - return None - else: - return resp.text - else: - print(general.warning(f"Something happened. keyserver.io returned status '{resp.status_code}' for '{query}'.")) - return None - - def do_mailfy(self, query, **kwargs): - """Verifying a mailfy query in this platform - - This might be redefined in any class inheriting from Platform. The only - condition is that any of this should return an equivalent array. - - Args: - query: The element to be searched. - - Returns: - A list of elements to be appended. A sample output format is as follows: - [ - { - "attributes": [ - { - "attributes": [], - "type": "com.i3visio.Email", - "value": "contacto@i3visio.com" - }, - { - "attributes": [], - "type": "com.i3visio.Alias", - "value": "contacto" - }, - { - "attributes": [], - "type": "com.i3visio.Domain", - "value": "i3visio.com" - }, - { - "attributes": [], - "type": "com.i3visio.Platform", - "value": "Twitter" - } - ], - "type": "com.i3visio.Profile", - "value": "Twitter - contacto@i3visio.com" - } - ] - """ - info = self.check_mailfy(query, **kwargs) - - results = [] - if info: - emails = set(re.findall(' <([^\&]+)>', info)) - - for i, email in enumerate(emails): - try: - expandedEntities = general.expand_entities_from_email(email) - r = { - "type": "com.i3visio.Profile", - "value": self.platformName + " - " + email, - "attributes": expandedEntities + [ - { - "type": "com.i3visio.Platform", - "value": self.platformName, - "attributes": [] - } - ] - } - results.append(r) - except IndexError: - # A result does not contain a @. - pass - - return results diff --git a/osrframework/wrappers/listaspam.py b/osrframework/wrappers/listaspam.py index 580593c..cfc0de1 100644 --- a/osrframework/wrappers/listaspam.py +++ b/osrframework/wrappers/listaspam.py @@ -18,7 +18,7 @@ ################################################################################ __author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" +__version__ = "3.0" from osrframework.utils.platforms import Platform @@ -30,73 +30,16 @@ self.platformName = "Listaspam" self.tags = ["phone"] - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = True - self.isValidMode["usufy"] = False - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - self.url = {} - self.url["phonefy"] = "http://www.listaspam.com/busca.php?Telefono=" - #self.url["usufy"] = "http://anyurl.com/user/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - self.validQuery["phonefy"] = ".+" - self.validQuery["usufy"] = ".+" - self.validQuery["searchfy"] = ".+" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the phone number is not appearing - self.notFoundText = {} - self.notFoundText["phonefy"] = [ - "No te quedes sin saber quien te llama por teléfono." - ] - #self.notFoundText["usufy"] = [] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - # Definition of regular expressions to be searched in phonefy mode - self.fieldsRegExp["phonefy"] = {} - self.fieldsRegExp["phonefy"]["i3visio.location.province"] = "(.*)," - self.fieldsRegExp["phonefy"]["i3visio.location.country"] = "class='country_located' alt='([a-zA-Zñ]*)'" - self.fieldsRegExp["phonefy"]["i3visio.text"] = '

(.*)

' - - # Definition of regular expressions to be searched in usufy mode - #self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} + self.modes = { + "phonefy": { + "debug": False, + "extra_fields": { + "com.i3visio.Location": "
📍([^<]+)
", + "com.i3visio.Labels": 'Nube de tags: (.+)' + }, + "needs_credentials": False, + "not_found_text": "No te quedes sin saber quién te llama por teléfono.", # Text that indicates a missing profile + "query_validator": "[0-9+\-\.]{6-16}", # Regular expression that the alias SHOULD match + "url": "http://www.listaspam.com/busca.php?Telefono={placeholder}", # Target URL where {placeholder} would be modified by the alias + } + } diff --git a/osrframework/wrappers/minds.py b/osrframework/wrappers/minds.py index ba24e0c..ee56681 100644 --- a/osrframework/wrappers/minds.py +++ b/osrframework/wrappers/minds.py @@ -18,7 +18,7 @@ ################################################################################ __author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" +__version__ = "3.0" from osrframework.utils.platforms import Platform @@ -37,11 +37,11 @@ "usufy": { "debug": False, "extra_fields": { - "com.i3visio.Name": '

([^<]+)

', # Regular expresion to extract the alias + "com.i3visio.Name": '

([^<]+)

', }, "needs_credentials": False, - "not_found_text": "Minds", # Text that indicates a missing profile - "query_validator": "[a-z0-9A-Z_]+", # Regular expression that the alias SHOULD match - "url": "https://www.minds.com/{placeholder}", # Target URL where {placeholder} would be modified by the alias + "not_found_text": "Sorry, the channel couldn't be found", + "query_validator": "[a-z0-9A-Z_]+", + "url": "https://www.minds.com/{placeholder}", } } diff --git a/osrframework/wrappers/okcupid.py b/osrframework/wrappers/okcupid.py index 2483e04..bfe2552 100644 --- a/osrframework/wrappers/okcupid.py +++ b/osrframework/wrappers/okcupid.py @@ -18,83 +18,55 @@ ################################################################################ __author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" +__version__ = "3.0" from osrframework.utils.platforms import Platform -class Okcupid(Platform): - """A object for Okcupid""" +class OkCupid(Platform): + """ class""" def __init__(self): - self.platformName = "Okcupid" + """Constructor with parameters + + This method permits the developer to instantiate dinamically Platform + objects.""" + self.platformName = "OkCupid" self.tags = ["contact"] + self.modes = { + "mailfy": { + "debug": False, + } + } - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False + def check_mailfy(self, query, kwargs={}): + """Verifying a mailfy query in this platform - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://www.okcupid.com/profile/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" + This might be redefined in any class inheriting from Platform. The only + condition is that any of this should return a dictionary as defined. - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False + Args: + query (str): The element to be searched. - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" + Returns: + String. The collected data if exists or None if not. + """ + import requests - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["Account not found"] - #self.notFoundText["searchfy"] = [] + s = requests.Session() - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} + # Getting the first response to grab the csrf_token + r1 = s.get('https://www.okcupid.com') - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" + # Launching the query to Instagram + r2 = s.post( + 'https://www.okcupid.com/1/apitun/signup/check_email', + json={"email": query}, + ) - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" + if self.modes["mailfy"]["debug"]: + print(f"Response: {[r2.text]}") - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} + if '"is_valid" : false' in r2.text: + return r2.text + return None diff --git a/osrframework/wrappers/onename.py b/osrframework/wrappers/onename.py index 9c5679d..6b23bb2 100644 --- a/osrframework/wrappers/onename.py +++ b/osrframework/wrappers/onename.py @@ -18,83 +18,29 @@ ################################################################################ __author__ = "Felix Brezo, Yaiza Rubio <contacto@i3visio.com>" -__version__ = "2.0" +__version__ = "3.0" from osrframework.utils.platforms import Platform class Onename(Platform): - """A <Platform> object for Onename""" + """<Platform> class""" def __init__(self): self.platformName = "Onename" self.tags = ["cryptocurrencies", "identity"] - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "<phonefy>" - self.url["usufy"] = "https://onename.com/" + "<usufy>" - #self.url["searchfy"] = "http://anyurl.com/search/" + "<searchfy>" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["User data not properly formatted"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} + self.modes = { + "usufy": { + "debug": False, + "extra_fields": {}, + "needs_credentials": False, + "not_found_text": "<title>Page Not Found", # Text that indicates a missing profile + "query_validator": "[a-zA-Z\.0-9_\-]+", # Regular expression that the alias SHOULD match + "url": "https://onename.com//{placeholder}", # Target URL where {placeholder} would be modified by the alias + "test": { + "valid": "james", + "invalid": "7ddf32e17a6ac5" + } + } + } diff --git a/osrframework/wrappers/pixinsight.py b/osrframework/wrappers/pixinsight.py deleted file mode 100644 index 3ea8cdb..0000000 --- a/osrframework/wrappers/pixinsight.py +++ /dev/null @@ -1,102 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Pixinsight(Platform): - """A object for Pixinsight""" - def __init__(self): - self.platformName = "Pixinsight" - self.tags = ["imagery"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "http://pixinsight.com/forum/index.php?action=profile;user=" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = [ - "The user whose profile you are trying to view does not exist." - ] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/sarahah.py b/osrframework/wrappers/sarahah.py deleted file mode 100644 index e21bede..0000000 --- a/osrframework/wrappers/sarahah.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Sarahah(Platform): - """A object for Sarahah""" - def __init__(self): - self.platformName = "Sarahah" - self.tags = ["social", "teenagers"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://.sarahah.com" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ['Sarahah '] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = {"start": 'Location: "', "end": '"'} - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/sidereel.py b/osrframework/wrappers/sidereel.py deleted file mode 100644 index 33b3417..0000000 --- a/osrframework/wrappers/sidereel.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Sidereel(Platform): - """A object for Sidereel""" - def __init__(self): - self.platformName = "Sidereel" - self.tags = ["social"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "http://www.sidereel.com/profile/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["Page Not Found"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/soup.py b/osrframework/wrappers/soup.py deleted file mode 100644 index c476074..0000000 --- a/osrframework/wrappers/soup.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Soup(Platform): - """A object for Soup""" - def __init__(self): - self.platformName = "Soup" - self.tags = ["social"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://" + "" + ".soup.io" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["

Happy happy joy joy! A new user!

"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/taringa.py b/osrframework/wrappers/taringa.py deleted file mode 100644 index bf91854..0000000 --- a/osrframework/wrappers/taringa.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Taringa(Platform): - """A object for Taringa""" - def __init__(self): - self.platformName = "Taringa" - self.tags = ["social"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "https://www.taringa.net/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ['/dist/404Retina.png', '>Taringa! - Inteligencia Colectiva en Taringa!', '>Taringa! - Inteligencia Colectiva en Taringa!' ] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/thesims.py b/osrframework/wrappers/thesims.py deleted file mode 100644 index 720fda4..0000000 --- a/osrframework/wrappers/thesims.py +++ /dev/null @@ -1,100 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -from osrframework.utils.platforms import Platform - - -class Thesims(Platform): - """A object for Thesims""" - def __init__(self): - self.platformName = "Thesims" - self.tags = ["gaming"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "http://mypage.thesims3.com/mypage/" + "" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = [" - Community - The Sims 3"] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/twitter.py b/osrframework/wrappers/twitter.py index 9763b07..b2bc583 100644 --- a/osrframework/wrappers/twitter.py +++ b/osrframework/wrappers/twitter.py @@ -32,103 +32,39 @@ self.tags = ["contact", "microblogging", "social"] # Base URL - self.baseURL = "http://twitter.com/" + self.baseURL = "https://twitter.com/" - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["mailfy"] = True - self.isValidMode["phonefy"] = False - self.isValidMode["searchfy"] = True - self.isValidMode["usufy"] = True - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - self.url["mailfy"] = "https://api.twitter.com/i/users/email_available.json?email=" - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["searchfy"] = "https://twitter.com/search?f=users&vertical=default&q=\"" + "" + "\"" - self.url["usufy"] = "http://twitter.com/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - self.needsCredentials["mailfy"] = False - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query. - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = "[a-zA-Z0-9_]+" - self.validQuery["searchfy"] = ".+" - self.validQuery["mailfy"] = ".+" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - self.notFoundText["mailfy"] = ['"valid":true'] - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["
\n ", "end": "\n

"} - #self.fieldsRegExp["usufy"]["ProfileHeaderCard-bio"] = {"start": "

", "end": "

"} - self.fieldsRegExp["usufy"]["i3visio.location"] = {"start": "\n ", "end": "\n \n \n\n
Se uni"} - self.fieldsRegExp["usufy"]["i3visio.uri.homepage"] = {"start": ""} - #self.fieldsRegExp["usufy"]["PhotoRail-headingText"] = {"start": "class=\"js-nav\">\n \n ", "end": " "} - - # Definition of regular expressions to be searched in searchfy mode - self.fieldsRegExp["searchfy"] = {} - self.searchfyAliasRegexp = "data-screen-name=\"([^\"]+)\"" - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} - + self.modes = { + "usufy": { + "debug": False, + "extra_fields": { + "com.i3visio.FullName": ' \((.+)\)', # Regular expresion to extract the alias + }, + "needs_credentials": False, + "not_found_text": "Sorry, that page does not exist. ", # Text that indicates a missing profile + "query_validator": "[a-zA-Z0-9_]{3,15}", # Regular expression that the alias SHOULD match + "url": "https://tweettunnel.com/{placeholder}", # Target URL where {placeholder} would be modified by the alias + }, + # Reimplementation needed of check_mailfy + "mailfy": { + "debug": False, + "extra_fields": {}, + "needs_credentials": False, + "not_found_text": '"valid":true', + "query_validator": ".+", + "url": "https://api.twitter.com/i/users/email_available.json?email=", + }, + } def do_usufy(self, query, **kwargs): - """ - Verifying a usufy query in this platform. + """Verifying a usufy query in this platform using the API - This might be redefined in any class inheriting from Platform. + If no credentials are provided, the standard verifier will be raised. Args: - ----- query: The element to be searched. Return: - ------- A list of elements to be appended. """ # Trying to interact with the API Wrapper @@ -151,9 +87,9 @@ return super(Twitter, self).do_usufy(query, **kwargs) def do_searchfy(self, query, **kwargs): - """Verifying a usufy query in this platform + """Verifying a searchfy query in this platform using the API - This might be redefined in any class inheriting from Platform. + If no credentials are provided, the standard verifier will be raised. Args: query (str): The element to be searched. diff --git a/osrframework/wrappers/v7n.py b/osrframework/wrappers/v7n.py deleted file mode 100644 index 309a741..0000000 --- a/osrframework/wrappers/v7n.py +++ /dev/null @@ -1,101 +0,0 @@ -################################################################################ -# -# Copyright 2015-2020 Félix Brezo and Yaiza Rubio -# -# This program is part of OSRFramework. You can redistribute it and/or modify -# it under the terms of the GNU Affero General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU Affero General Public License for more details. -# -# You should have received a copy of the GNU Affero General Public License -# along with this program. If not, see . -# -################################################################################ - -__author__ = "Felix Brezo, Yaiza Rubio " -__version__ = "2.0" - - -import osrframework.utils.browser as browser -from osrframework.utils.platforms import Platform - - -class V7n(Platform): - """A object for V7n""" - def __init__(self): - self.platformName = "V7n" - self.tags = ["development"] - - ######################## - # Defining valid modes # - ######################## - self.isValidMode = {} - self.isValidMode["phonefy"] = False - self.isValidMode["usufy"] = True - self.isValidMode["searchfy"] = False - - ###################################### - # Search URL for the different modes # - ###################################### - # Strings with the URL for each and every mode - self.url = {} - #self.url["phonefy"] = "http://anyurl.com//phone/" + "" - self.url["usufy"] = "http://www.v7n.com/forums/members/" + "" + ".html" - #self.url["searchfy"] = "http://anyurl.com/search/" + "" - - ###################################### - # Whether the user needs credentials # - ###################################### - self.needsCredentials = {} - #self.needsCredentials["phonefy"] = False - self.needsCredentials["usufy"] = False - #self.needsCredentials["searchfy"] = False - - ################# - # Valid queries # - ################# - # Strings that will imply that the query number is not appearing - self.validQuery = {} - # The regular expression '.+' will match any query - #self.validQuery["phonefy"] = ".*" - self.validQuery["usufy"] = ".+" - #self.validQuery["searchfy"] = ".*" - - ################### - # Not_found clues # - ################### - # Strings that will imply that the query number is not appearing - self.notFoundText = {} - #self.notFoundText["phonefy"] = [] - self.notFoundText["usufy"] = ["This user has not registered and therefore does not have a profile to view."] - #self.notFoundText["searchfy"] = [] - - ######################### - # Fields to be searched # - ######################### - self.fieldsRegExp = {} - - # Definition of regular expressions to be searched in phonefy mode - #self.fieldsRegExp["phonefy"] = {} - # Example of fields: - #self.fieldsRegExp["phonefy"]["i3visio.location"] = "" - - # Definition of regular expressions to be searched in usufy mode - self.fieldsRegExp["usufy"] = {} - # Example of fields: - #self.fieldsRegExp["usufy"]["i3visio.location"] = "" - # Definition of regular expressions to be searched in searchfy mode - #self.fieldsRegExp["searchfy"] = {} - # Example of fields: - #self.fieldsRegExp["searchfy"]["i3visio.location"] = "" - - ################ - # Fields found # - ################ - # This attribute will be feeded when running the program. - self.foundFields = {} diff --git a/osrframework/wrappers/youtube.py b/osrframework/wrappers/youtube.py index df809b6..29b6ff3 100644 --- a/osrframework/wrappers/youtube.py +++ b/osrframework/wrappers/youtube.py @@ -40,7 +40,7 @@ "com.i3visio.Date.Create": '{"start": "
  • ", "end": "
    • "}', # Regular expresion to extract the alias }, "needs_credentials": False, - "not_found_text": "channel-empty-message banner-message", # Text that indicates a missing profile + "not_found_text": "error-page", # Text that indicates a missing profile "query_validator": "[^@, ]+", # Regular expression that the alias SHOULD match "url": "https://www.youtube.com/user/{placeholder}/about", # Target URL where {placeholder} would be modified by the alias }, diff --git a/osrframework.egg-info/PKG-INFO b/osrframework.egg-info/PKG-INFO index dc0c808..da9c05d 100644 --- a/osrframework.egg-info/PKG-INFO +++ b/osrframework.egg-info/PKG-INFO @@ -1,114 +1,12 @@ Metadata-Version: 2.1 Name: osrframework -Version: 0.20.1 +Version: 0.20.2 Summary: OSRFramework - A set of AGPLv3+ OSINT tools developed by i3visio analysts for online research. Home-page: http://github.com/i3visio/osrframework Author: Felix Brezo and Yaiza Rubio Author-email: contacto@i3visio.com License: COPYING -Description: OSRFramework - ============ - - OSRFramework: Open Sources Research Framework - - Copyright (C) 2014-2020 F. Brezo and Y. Rubio, i3visio - - [![Version in PyPI](https://img.shields.io/pypi/v/osrframework.svg)]() - [![License](https://img.shields.io/badge/license-GNU%20Affero%20General%20Public%20License%20Version%203%20or%20Later-blue.svg)]() - - 1 - Description - --------------- - - OSRFramework is a GNU AGPLv3+ set of libraries developed by i3visio to perform Open Source Intelligence collection tasks. - They include references to a bunch of different applications related to username checking, DNS lookups, information leaks research, deep web search, regular expressions extraction and many others. - At the same time, by means of ad-hoc Maltego transforms, OSRFramework provides a way of making these queries graphically as well as several interfaces to interact with like OSRFConsole or a Web interface. - - 2 - License: GNU AGPLv3+ - ------------------------ - - This is free software, and you are welcome to redistribute it under certain conditions. - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU Affero General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . - - - For more details on this issue, check the [COPYING](COPYING) file. - - 3 - Installation - ---------------- - - Fast way to do it on any system for a user with administration privileges: - ``` - pip3 install osrframework - ``` - You can upgrade to the latest release of the framework with: - ``` - pip3 install osrframework --upgrade - ``` - This will manage all the dependencies for you and install the latest version of the framework. - - If you needed further information on how to install OSRFramework on certain systems, note that you may need to add `export PATH=$PATH:$HOME/.local/bin` to your `~/.bashrc_profile`). This has been confirmed in some distributions, including MacOS. - In any case, we recommend you yo have a look at the [INSTALL.md](doc/INSTALL.md) file where we provide additional details for these cases. - - 4 - Basic usage - --------------- - - If everything went correctly (we hope so!), it's time for trying usufy., mailfy and so on. - But we are they - ? They are installed in your path meaning that you can open a terminal anywhere and typing the name of the program (seems to be an improvement from previous installations...). Examples: - ``` - osrf --help - usufy -n i3visio febrezo yrubiosec -p twitter facebook - searchfy -q "i3visio" - mailfy -n i3visio - ``` - - Type -h or --help to get more information about which are the parameters of each application. - - The tools installed in this package include: - - - `alais_generator`. Generates candidate nicknames based on known info about the target. **Input**: information about the target. **Output**: list of possible nicknames. - - `checkfy`. Guesses possible emails based on a list of candidate nicknames and a pattern. **Input**: list of nicknames and an email pattern. **Output**. list of emails matching the pattern.. - - `domainfy`. Finds domains that currently resolve using a given word or nickname. **Input**: liat of words. **Output**: domains using that word that currently resolve. - - `mailfy`. Find more information about emails taken as a reference either a nickname (to generate a list of possible emails) or the email list. **Input**: list of nicknames or emails. **Output**: found information about the email. - - `osrf`. Shared wrapper for the rest of the applications. All commands can also be used as `osrf usufy…`, `osrf mailfy…`, etc. - - `phonefy`. Recovers information about mobile phones linked to known spam practices. **Inputs**: list of phones. **Outputs**: Phones linked to spam. - - `searchfy`. Finds profiles linked to a fullname. **Inputs**: list of phones. **Outputs**: Known profiles linked to the query. - - `usufy`. Identifies socialmedia profiles using a given nickname. **Inputs**: list of nicknames. **Outputs**: Known profiles in socialmedia using those nicknames. - - You can find the configuration files in a folder created in your user home to define the default behaviour of the applications: - ``` - # Configuration files for Linux and MacOS - ~/.config/OSRFramework/ - # Configuration files for Windows - C:\Users\\OSRFramework\ - ``` - - OSRFramework will look for the configuration settings for each application stored there. - You can add new credentials there and if something goes wrong, you can always restore the files stored in the `defaults` subfolder. - - If you are experiencing problems, you might fight relevant information in the (FAQ Section)[doc/FAQ.md]. - - 5 - HACKING - ----------- - - If you want to extend the functionalities of OSRFramework and you do not know where to start from, check the [HACKING.md](doc/HACKING.md) file. - - 6 - AUTHORS - ----------- - - More details about the authors in the [AUTHORS.md](AUTHORS.md) file. - +Description: UNKNOWN Keywords: python osint harvesting profiling username socialmedia forums Platform: UNKNOWN Classifier: Development Status :: 4 - Beta diff --git a/osrframework.egg-info/SOURCES.txt b/osrframework.egg-info/SOURCES.txt index f3fe2f3..63c5ed3 100644 --- a/osrframework.egg-info/SOURCES.txt +++ b/osrframework.egg-info/SOURCES.txt @@ -107,7 +107,6 @@ osrframework/wrappers/bandcamp.py osrframework/wrappers/bennugd.py osrframework/wrappers/betblog.py -osrframework/wrappers/bitbacker.py osrframework/wrappers/bitbucket.py osrframework/wrappers/bitcointalk.py osrframework/wrappers/bitrated.py @@ -116,10 +115,8 @@ osrframework/wrappers/blogspot.py osrframework/wrappers/boonex.py osrframework/wrappers/bubok.py -osrframework/wrappers/bucketlistly.py osrframework/wrappers/buddypic.py osrframework/wrappers/burbuja.py -osrframework/wrappers/canva.py osrframework/wrappers/carbonmade.py osrframework/wrappers/cartodb.py osrframework/wrappers/causes.py @@ -130,7 +127,6 @@ osrframework/wrappers/codecademy.py osrframework/wrappers/codementor.py osrframework/wrappers/coderwall.py -osrframework/wrappers/colourlovers.py osrframework/wrappers/connectingsingles.py osrframework/wrappers/couchsurfing.py osrframework/wrappers/crokes.py @@ -144,7 +140,6 @@ osrframework/wrappers/disqus.py osrframework/wrappers/doodle.py osrframework/wrappers/douban.py -osrframework/wrappers/dreamstime.py osrframework/wrappers/dribbble.py osrframework/wrappers/drupal.py osrframework/wrappers/dzone.py @@ -171,6 +166,7 @@ osrframework/wrappers/freerepublic.py osrframework/wrappers/getsatisfaction.py osrframework/wrappers/github.py +osrframework/wrappers/gnupgkeys.py osrframework/wrappers/goblinrefuge.py osrframework/wrappers/goodreads.py osrframework/wrappers/gravatar.py @@ -189,7 +185,6 @@ osrframework/wrappers/jamiiforums.py osrframework/wrappers/kali.py osrframework/wrappers/kanogames.py -osrframework/wrappers/key_server.py osrframework/wrappers/keybase.py osrframework/wrappers/kickstarter.py osrframework/wrappers/kinja.py @@ -235,7 +230,6 @@ osrframework/wrappers/periscope.py osrframework/wrappers/phishtank.py osrframework/wrappers/photobucket.py -osrframework/wrappers/pixinsight.py osrframework/wrappers/pixls.py osrframework/wrappers/pjrc.py osrframework/wrappers/pokerred.py @@ -250,18 +244,15 @@ osrframework/wrappers/ripenear.py osrframework/wrappers/rojadirecta.py osrframework/wrappers/ruby.py -osrframework/wrappers/sarahah.py osrframework/wrappers/scribd.py osrframework/wrappers/seatwish.py osrframework/wrappers/sencha.py -osrframework/wrappers/sidereel.py osrframework/wrappers/singletrackworld.py osrframework/wrappers/slashdot.py osrframework/wrappers/slideshare.py osrframework/wrappers/smartcitizen.py osrframework/wrappers/smugmug.py osrframework/wrappers/soundcloud.py -osrframework/wrappers/soup.py osrframework/wrappers/spaniards.py osrframework/wrappers/spoj.py osrframework/wrappers/spotify.py @@ -269,10 +260,8 @@ osrframework/wrappers/steamcommunity.py osrframework/wrappers/steemit.py osrframework/wrappers/steinberg.py -osrframework/wrappers/taringa.py osrframework/wrappers/teamtreehouse.py osrframework/wrappers/telegram.py -osrframework/wrappers/thesims.py osrframework/wrappers/thestudentroom.py osrframework/wrappers/theverge.py osrframework/wrappers/thiscrush.py @@ -282,7 +271,6 @@ osrframework/wrappers/twoplustwo.py osrframework/wrappers/typepad.py osrframework/wrappers/unsplash.py -osrframework/wrappers/v7n.py osrframework/wrappers/venmo.py osrframework/wrappers/verbling.py osrframework/wrappers/vexforum.py diff --git a/osrframework.egg-info/requires.txt b/osrframework.egg-info/requires.txt index d3c07e2..95377ab 100644 --- a/osrframework.egg-info/requires.txt +++ b/osrframework.egg-info/requires.txt @@ -19,4 +19,3 @@ colorama configparser cfscrape -pip>=19.0.0 diff --git a/requirements.txt b/requirements.txt index d3c07e2..95377ab 100644 --- a/requirements.txt +++ b/requirements.txt @@ -19,4 +19,3 @@ colorama configparser cfscrape -pip>=19.0.0 diff --git a/setup.py b/setup.py index 0f05c1c..e71c57e 100644 --- a/setup.py +++ b/setup.py @@ -50,8 +50,8 @@ requirements = file.read().splitlines() # Depending on the place in which the project is going to be upgraded -def read(fname): - return open(os.path.join(os.path.dirname(__file__), fname)).read() +"""def read(fname): + return open(os.path.join(os.path.dirname(__file__), fname)).read()""" # Creating the application paths paths = configuration.get_config_path() @@ -105,7 +105,7 @@ 'Topic :: Text Processing :: Markup :: HTML' ], packages=find_packages(), - long_description=read('README.md'), + #long_description=read('README.md'), long_description_content_type="text/markdown", install_requires=requirements, )