a D=]ip@sddlZddlZddlZddlmZddlmZddlmZm Z zddl Z dZ Wne yfdZ Yn0zddl Z dZWne ydZYn0GdddZdS) N)getpass)_sos) is_executableTIMEOUT_DEFAULTTFc@seZdZdZdZdZdZdZdZdZ dZ dZ dZ dZ dZdZdZdZdZdZdZdZdZdZdZdZdZdZdZddd gddddd d d dddddddd ZdSd d ZddZddZ e!ddZ"ddZ#ddZ$ddZ%ddZ&ddZ'ddZ(d d!Z)d"d#Z*d$d%Z+d&d'Z,d(d)Z-dTd*d+Z.d,d-Z/d.d/Z0d0d1Z1d2d3Z2d4d5Z3d6d7Z4d8d9Z5d:d;Z6dd?Z8d@dAZ9dUdBdCZ:dDdEZ;dVdGdHZdMdNZ?dXdOdPZ@dYdQdRZAdS)Z UploadTargetz This class is designed to upload files to a distribution defined location. These files can be either sos reports, sos collections, or other kind of files like: vmcores, application cores, logs, etc. z Upload a file (can be an sos report, a must-gather, or others) to a distribution defined remote location zGeneric UploadZgenericN/zhttps://s3.amazonaws.comFauto) upload_filecase_id low_priorityZprofiles upload_urlupload_directory upload_user upload_pass upload_methodupload_no_ssl_verifyupload_protocolupload_s3_endpointupload_s3_regionupload_s3_bucketupload_s3_access_keyupload_s3_secret_keyupload_s3_object_prefix upload_targetcCs"td|_||_||_||_dS)NZsos_ui)loggingZ getLoggerui_logparsercmdlineargs)selfrrrr!?/usr/lib/python3.9/site-packages/sos/upload/targets/__init__.py__init__as zUploadTarget.__init__cCsdS)aSThis should be overridden by upload targets This is called by sos upload on each target type that exists, and is meant to return True when the upload target matches a criteria that indicates that is the local upload target that should be used. Only the first upload target to determine a match is selectedFr!r r!r!r"check_distributionhszUploadTarget.check_distributioncCs|jS)N)upload_target_idr$r!r!r" get_target_idrszUploadTarget.get_target_idcCs|jr |jS|jS)z-Returns the upload target's name as a string.)upload_target_name__name__lower)clsr!r!r"nameuszUploadTarget.namecCs*|jd|jd|jdj|jdjdS)N cmdlineoptspolicy)r-r.r r) hook_commonsr rr$r!r!r" get_commons|s   zUploadTarget.get_commonscCs ||_dS)zMSet common host data for the Upload targets to reference N)commons)r r1r!r!r" set_commonsszUploadTarget.set_commonscCs||_||_|jd}|jd}|jr2||j|_|j|_|j|_|j|_ d|_ |j |_ |j |_ |j |_ |j|_|j|_|j|_|js|js|r|jdks||n*|jdkr|||||jddS)Nr-r.rs3)r/r0r1r Z_configure_low_priorityr rrrupload_passwordupload_archive_namerrrrrrZbatchquietget_upload_urlrprompt_for_upload_userprompt_for_upload_passwordprompt_for_upload_s3_bucketprompt_for_upload_s3_endpointprompt_for_upload_s3_access_keyprompt_for_upload_s3_secret_keyrinfo)r r/Z cmdline_optsr.r!r!r"pre_works>     zUploadTarget.pre_workcCs4|s0d|d|d}tt||_dS)zuShould be overridden by targets to determine if an access key needs to be provided for upload or not z0Please provide the upload access key for bucket  via endpoint : N)get_upload_s3_access_keyget_upload_s3_bucketget_upload_s3_endpointinput_rr msgr!r!r"r<sz,UploadTarget.prompt_for_upload_s3_access_keycCs0|s,d|d|d}t||_dS)ztShould be overridden by targets to determine if a secret key needs to be provided for upload or not z0Please provide the upload secret key for bucket r@rAN)get_upload_s3_secret_keyrCrDrrrGr!r!r"r=sz,UploadTarget.prompt_for_upload_s3_secret_keycCsH|jsB|jr*|jdr*|jdd|_nttd}|d|_|jS)zpShould be overridden by targets to determine if a bucket needs to be provided for upload or not s3://Nz"Please provide the upload bucket: r)rr startswithrErFstrip)r user_inputr!r!r"r:s   z(UploadTarget.prompt_for_upload_s3_bucketcCs>|j}|js8d|d|d}tt|}|p4||_|jS)zsShould be overridden by targets to determine if an endpoint needs to be provided for upload or not z.Please provide the upload endpoint for bucket z (default: z): )_upload_s3_endpointrrCrErF)r Zdefault_endpointrHrNr!r!r"r;s  z*UploadTarget.prompt_for_upload_s3_endpointcCs*|s&d|d}tt||_dS)zcShould be overridden by targets to determine if a user needs to be provided or not zPlease provide upload user for rAN)get_upload_userr7rErFrrGr!r!r"r8sz#UploadTarget.prompt_for_upload_usercCs4|s0||jkr0d|d}t||_dS)zrShould be overridden by targets to determine if a password needs to be provided for upload or not z'Please provide the upload password for rAN)get_upload_passwordrP _upload_userrr4rGr!r!r"r9sz'UploadTarget.prompt_for_upload_passwordcCsL||_|js||_|js$td|}|jtd||S)a  Entry point for sos attempts to upload the generated archive to a target or user specified location. Currently there is support for HTTPS, SFTP, and FTP. HTTPS uploads are preferred for target-defined defaults. Targets that need to override uploading methods should override the respective upload_https(), upload_sftp(), and/or upload_ftp() methods and should NOT override this method. :param archive: The archive filepath to use for upload :type archive: ``str`` In order to enable this for a target, that target needs to implement the following: Required Class Attrs :_upload_url: The default location to use. Note these MUST include protocol header :_upload_user: Default username, if any else None :_upload_password: Default password, if any else None The following Class Attrs may optionally be overidden by the Target :_upload_directory: Default FTP server directory, if any The following methods may be overridden by ``Target`` as needed `prompt_for_upload_user()` Determines if sos should prompt for a username or not. `get_upload_user()` Determines if the default or a different username should be used `get_upload_https_auth()` Format authentication data for HTTPS uploads `get_upload_url_string()` Print a more human-friendly string than vendor URLs zBNo upload destination provided by upload target or by --upload-urlzAttempting upload to ) r5r r7 Exception_determine_upload_typerr>rFget_upload_url_string)r archiveZ upload_funcr!r!r"upload_archives, zUploadTarget.upload_archivecCsv|j|j|j|jd}|jdj|vr6||jdjSd|jvrHtd|jd\}}||vrntd|||S)aBased on the url provided, determine what type of upload to attempt. Note that this requires users to provide a FQDN address, such as https://myvendor.com/api or ftp://myvendor.com instead of myvendor.com/api or myvendor.com )ftpsftphttpsr3r-z://z#Must provide protocol in upload URLz&Unsupported or unrecognized protocol: ) upload_ftp upload_sftp upload_https upload_s3r1rr rSsplit)r ZprotsZprotrFr!r!r"rT3s z#UploadTarget._determine_upload_typecCs&|s |}|s|}tj||S)ahFormats the user/password credentials using basic auth :param user: The username for upload :type user: ``str`` :param password: Password for `user` to use for upload :type password: ``str`` :returns: The user/password auth suitable for use in requests calls :rtype: ``requests.auth.HTTPBasicAuth()`` )rPrQrequestsauthZ HTTPBasicAuth)r userpasswordr!r!r"get_upload_https_authIs z"UploadTarget.get_upload_https_authcCstddp|jp|jS)zHelper function to determine if we should use the target default upload access key or one provided by the user :returns: The access_key to use for upload :rtype: ``str`` ZSOSUPLOADS3ACCESSKEYN)osgetenvr_upload_s3_access_keyr$r!r!r"rB\s  z%UploadTarget.get_upload_s3_access_keycCs|js||jS)zHelper function to determine if we should use the target default upload endpoint or one provided by the user :returns: The S3 Endpoint to use for upload :rtype: ``str`` )rr;r$r!r!r"rDgsz#UploadTarget.get_upload_s3_endpointcCs |jp |jS)zHelper function to determine if we should use the target default upload region or one provided by the user :returns: The S3 region to use for upload :rtype: ``str`` )r_upload_s3_regionr$r!r!r"get_upload_s3_regionrsz!UploadTarget.get_upload_s3_regioncCsb|jrH|jdrH|jdddd}|d|_t|dkrH|d|_|jsV||jp`|jS)zHelper function to determine if we should use the target default upload bucket or one provided by the user :returns: The S3 bucket to use for upload :rtype: ``str`` rJrKNrr)r rLr_rlenrr:_upload_s3_bucket)r Zbucket_and_prefixr!r!r"rC{s   z!UploadTarget.get_upload_s3_bucketcCs |jp |jS)zHelper function to determine if we should use the target default upload object prefix or one provided by the user :returns: The S3 object prefix to use for upload :rtype: ``str`` )r_upload_s3_object_prefixr$r!r!r"get_upload_s3_object_prefixsz(UploadTarget.get_upload_s3_object_prefixcCstddp|jp|jS)zHelper function to determine if we should use the target default upload secret key or one provided by the user :returns: The S3 secret key to use for upload :rtype: ``str`` ZSOSUPLOADS3SECRETKEYN)rerfr_upload_s3_secret_keyr$r!r!r"rIs  z%UploadTarget.get_upload_s3_secret_keycCsF|js:|jr:|jr:|jr:|}|}d|d||_|jpD|jS)zHelper function to determine if we should use the target default upload url or one provided by the user :returns: The URL to use for upload :rtype: ``str`` rJr)r rrrrCrn _upload_url)r bucketprefixr!r!r"r7szUploadTarget.get_upload_urlcCsd}t|d|}|S)Nz([^:]+://[^:]+:)([^@]+)(@.+)z \1********\3)resub)r urlpatternZobfuscated_urlr!r!r"_get_obfuscated_upload_urlsz'UploadTarget._get_obfuscated_upload_urlcCs||S)zUsed by upload targets to potentially change the string used to report upload location from the URL to a more human-friendly string )rwr7r$r!r!r"rUsz"UploadTarget.get_upload_url_stringcCstddp|jp|jS)zHelper function to determine if we should use the target default upload user or one provided by the user :returns: The username to use for upload :rtype: ``str`` Z SOSUPLOADUSERN)rerfrrRr$r!r!r"rPs  zUploadTarget.get_upload_usercCstddp|jp|jS)a_Helper function to determine if we should use the target default upload password or one provided by the user A user provided password, either via option or the 'SOSUPLOADPASSWORD' environment variable will have precendent over any target value :returns: The password to use for upload :rtype: ``str`` ZSOSUPLOADPASSWORDN)rerfr4_upload_passwordr$r!r!r"rQs z UploadTarget.get_upload_passwordc Cs<tdstdz ddl}Wn.tyJ}ztd|WYd}~n d}~00d}|s\|}|sh|}|dd}d |d |}|j|d d }d dd|j |j g} |j | dd} | dkrd}n| dkr| |d d|j |j g} |j | dddk}|st| td|nX| dkrz password:zConnection refusedtimeoutTrjzPermission denied z#Incorrect username or password for zConnection refused by z. Incorrect port?z!Timeout hit trying to connect to z,Unexpected error trying to connect to sftp: zUnable to connect via SFTP to zput  z100%zNo such file or directoryZbyezTimeout expired while uploadingzUnknown error during upload: z&Unable to write archive to destinationz!Unexpected response from server: )rrSpexpect ImportErrorrPrQr7replaceZspawnZTIMEOUTZEOFexpectZsendlinecloserUbeforer5_get_sftp_upload_name)r rbrcrerrZsftp_connectedZsftp_urlZsftp_cmdretZ sftp_expectsidxZ pass_expectsZput_cmdZ put_expectsZ put_successr!r!r"r\s                 zUploadTarget.upload_sftpcCs*|jdd}|jr&tj|j|}|S)a$If a specific file name pattern is required by the SFTP server, override this method in the relevant Upload Target. Otherwise the archive's name on disk will be used :returns: Filename as it will exist on the SFTP server :rtype: ``str`` r)r5r_rrepathjoin)r fnamer!r!r"r7sz"UploadTarget._get_sftp_upload_nameTcCstj||||tdS)zIf upload_https() needs to use requests.put(), use this method. Targets should override this method instead of the base upload_https() :param archive: The open archive file object )dataraverifyr})r`putr7rdr)r rVrr!r!r"_upload_https_putDs zUploadTarget._upload_https_putcCsiS)zJDefine any needed headers to be passed with the POST request here r!r$r!r!r"_get_upload_headersOsz UploadTarget._get_upload_headerscCs:d|jdd||fi}tj||||tdS)zIf upload_https() needs to use requests.post(), use this method. Targets should override this method instead of the base upload_https() :param archive: The open archive file object filerr)filesrarr})r,r_rr`Zpostr7rdr)r rVrrr!r!r"_upload_https_postTs zUploadTarget._upload_https_postcCsts tdt|jd}|jdjdkr2|j}n |jdj}|jdjdu}|dkrd|||}n | ||}|j dvr|j dkrtd td |j d |j Wd d S1s0Yd S)zAttempts to upload the archive to an HTTPS location. :returns: ``True`` if upload is successful :rtype: ``bool`` :raises: ``Exception`` if upload was unsuccessful z7Unable to upload due to missing python requests libraryrbr-r Fr)iz/Authentication failed: invalid user credentialszPOST request returned rANT) REQUESTS_LOADEDrSopenr5r1r_upload_methodrrrZ status_codereason)r Zarcmethodrrr!r!r"r]cs&     zUploadTarget.upload_httpsc Csddl}ddl}|s|}|dur,td|dd}|sD|}|sP|}|s`|jp^|j}z.|j |||dd}|std| |Wn|j y}ztd ||WYd}~nd}~0|j y}ztd ||WYd}~nd}~0|j y}zt|d} | d kr6td |d || dkrRtd|d || dkrltd||tdt||WYd}~n d}~00t|jd.} |d|jdd| Wdn1s0Y|dS)aTAttempts to upload the archive to either the target defined or user provided FTP location. :param url: The URL to upload to :type url: ``str`` :param directory: The directory on the FTP server to write to :type directory: ``str`` or ``None`` :param user: The user to authenticate with :type user: ``str`` :param password: The password to use for `user` :type password: ``str`` :returns: ``True`` if upload is successful :rtype: ``bool`` :raises: ``Exception`` if upload in unsuccessful rNzPno FTP server specified by upload target, use --upload-url to specify a locationzftp://rr{r|z3connection failed, did you set a user and password?z timeout hit while connecting to zunable to connect to Z503zcould not login as ''Z530zinvalid password for user 'Z550z"could not set upload directory to z#error trying to establish session: rzSTOR rrT)ftplibsocketr7rSrrPrQr_upload_directoryZFTPcwdr}ZgaierrorZ error_permstrr_rr5Z storbinaryquit) r ruZ directoryrbrcrrZsessionrerrnoZ_arcfiler!r!r"r[sV  $$   $zUploadTarget.upload_ftpc Csts td|s|}|s$|}|s6|d}|s|}|dkr`|dr`|dd}|dkr|ds|r|dnd}|s| }|s| }t j d||||d}z*||j dd}||j ||Wd Sty} z td t| | WYd} ~ n d} ~ 00dS) aAttempts to upload the archive to an S3 bucket. :param endpoint: The S3 endpoint to upload to :type endpoint: str :param region: The S3 region to upload to :type region: str :param bucket: The name of the S3 bucket to upload to :type bucket: str :param prefix: The prefix for the S3 object/key :type prefix: str :param access_key: The access key for the S3 bucket :type access_key: str :param secret_key: The secret key for the S3 bucket :type secret_key: str :returns: True if upload is successful :rtype: bool :raises: Exception if upload is unsuccessful z4Unable to upload due to missing python boto3 libraryrrrjNr3)Z endpoint_urlZ region_nameZaws_access_key_idZaws_secret_access_keyrTzFailed to upload to S3: ) BOTO3_LOADEDrSrDrirCrMrnrLendswithrBrIboto3Zclientr5r_r r) r ZendpointZregionrqrrZ access_keyZ secret_keyZ s3_clientkeyer!r!r"r^s> zUploadTarget.upload_s3)NNN)NN)NN)T)T)NNNN)NNNNNN)Br) __module__ __qualname____doc__Zdescr(r&Z _upload_filerprrRrxrrOrlrgrorhrmr rr4rrrrrrrZ arg_defaultsr#r%r' classmethodr,r0r2r?r<r=r:r;r8r9rWrTrdrBrDrirCrnrIr7rwrUrPrQr\rrrrr]r[r^r!r!r!r"r s     *    8        c   Fr)rersrrZsosrrFZ sos.utilitiesrrr`rrrrrr!r!r!r" s