Commit 0a705c27-cecb-4408-82a3-dcfb912c9731/upstream - python-webargs

+6

-5

.pre-commit-config.yaml less more

0	0	repos:
1	1	- repo: https://github.com/asottile/pyupgrade
2		rev: v2.7.3
	2	rev: v2.11.0
3	3	hooks:
4	4	- id: pyupgrade
5	5	args: ["--py36-plus"]

8	8	hooks:
9	9	- id: black
10	10	- repo: https://gitlab.com/pycqa/flake8
11		rev: 3.8.4
	11	rev: 3.9.0
12	12	hooks:
13	13	- id: flake8
14		additional_dependencies: [flake8-bugbear==20.1.0]
	14	additional_dependencies: [flake8-bugbear==21.4.3]
15	15	- repo: https://github.com/asottile/blacken-docs
16		rev: v1.8.0
	16	rev: v1.10.0
17	17	hooks:
18	18	- id: blacken-docs
19	19	additional_dependencies: [black==20.8b1]
20	20	args: ["--target-version", "py35"]
21	21	- repo: https://github.com/pre-commit/mirrors-mypy
22		rev: v0.790
	22	rev: v0.812
23	23	hooks:
24	24	- id: mypy
25	25	language_version: python3
26	26	files: ^src/webargs/
	27	additional_dependencies: ["marshmallow>=3,<4"]

+1

-0

AUTHORS.rst less more

50	50	* Lefteris Karapetsas `@lefterisjp <https://github.com/lefterisjp>`_
51	51	* Utku Gultopu `@ugultopu <https://github.com/ugultopu>`_
52	52	* Jason Williams `@jaswilli <https://github.com/jaswilli>`_
	53	* Grey Li `@greyli <https://github.com/greyli>`_

+26

-0

CHANGELOG.rst less more

0	0	Changelog
1	1	---------
	2
	3	8.0.0 (2021-04-08)
	4	******************
	5
	6	Features:
	7
	8	* Add `Parser.pre_load` as a method for allowing users to modify data before
	9	schema loading, but without redefining location loaders. See advanced docs on
	10	`Parser pre_load` for usage information
	11
	12	* ``unknown`` defaults to `None` for body locations (`json`, `form` and
	13	`json_or_form`) (:issue:`580`).
	14
	15	* Detection of fields as "multi-value" for unpacking lists from multi-dict
	16	types is now extensible with the ``is_multiple`` attribute. If a field sets
	17	``is_multiple = True`` it will be detected as a multi-value field.
	18	(:issue:`563`)
	19
	20	* If ``is_multiple`` is not set or is set to ``None``, webargs will check if the
	21	field is an instance of ``List`` or ``Tuple``.
	22
	23	* A new attribute on ``Parser`` objects, ``Parser.KNOWN_MULTI_FIELDS`` can be
	24	used to set fields which should be detected as ``is_multiple=True`` even when
	25	the attribute is not set.
	26
	27	See docs on "Multi-Field Detection" for more details.
2	28
3	29	7.0.1 (2020-12-14)
4	30	******************

+138

-8

docs/advanced.rst less more

109	109
110	110	@use_args(UserSchema())
111	111	def profile_view(args):
112		username = args["userame"]
	112	username = args["username"]
113	113	# ...
114	114
115	115

151	151	+++++++++++++++++
152	152
153	153	By default, webargs will pass `unknown=marshmallow.EXCLUDE` except when the
154		location is `json`, `form`, `json_or_form`, `path`, or `path`. In those cases,
155		it uses `unknown=marshmallow.RAISE` instead.
	154	location is `json`, `form`, `json_or_form`, or `path`. In those cases, it uses
	155	`unknown=marshmallow.RAISE` instead.
156	156
157	157	You can change these defaults by overriding `DEFAULT_UNKNOWN_BY_LOCATION`.
158	158	This is a mapping of locations to values to pass.

179	179	# so EXCLUDE will be used
180	180	@app.route("/", methods=["GET"])
181	181	@parser.use_args({"foo": fields.Int()}, location="query")
182		def get(self, args):
	182	def get(args):
183	183	return f"foo x 2 = {args['foo'] * 2}"
184	184
185	185

187	187	# so no value will be passed for `unknown`
188	188	@app.route("/", methods=["POST"])
189	189	@parser.use_args({"foo": fields.Int(), "bar": fields.Int()}, location="json")
190		def post(self, args):
	190	def post(args):
191	191	return f"foo x bar = {args['foo'] * args['bar']}"
192	192
193	193

204	204	# effect and `INCLUDE` will always be used
205	205	@app.route("/", methods=["POST"])
206	206	@parser.use_args({"foo": fields.Int(), "bar": fields.Int()}, location="json")
207		def post(self, args):
	207	def post(args):
208	208	unexpected_args = [k for k in args.keys() if k not in ("foo", "bar")]
209	209	return f"foo x bar = {args['foo'] * args['bar']}; unexpected args={unexpected_args}"
210	210

236	236	# as a result, the schema's behavior (EXCLUDE) is used
237	237	@app.route("/", methods=["POST"])
238	238	@use_args(RectangleSchema(), location="json", unknown=None)
239		def get(self, args):
	239	def get(args):
240	240	return f"area = {args['length'] * args['width']}"
241	241
242	242

274	274
275	275
276	276	@use_args(RectangleSchema)
277		def post(self, rect: Rectangle):
	277	def post(rect: Rectangle):
278	278	return f"Area: {rect.length * rect.width}"
279	279
280	280	Packages such as `marshmallow-sqlalchemy <https://github.com/marshmallow-code/marshmallow-sqlalchemy>`_ and `marshmallow-dataclass <https://github.com/lovasoa/marshmallow_dataclass>`_ generate schemas that deserialize to non-dictionary objects.

434	434	structure_dict_pair(r, k, v)
435	435	return r
436	436
	437	Parser pre_load
	438	---------------
	439
	440	Similar to ``@pre_load`` decorated hooks on marshmallow Schemas,
	441	:class:`Parser <webargs.core.Parser>` classes define a method,
	442	`pre_load <webargs.core.Parser.pre_load>` which can
	443	be overridden to provide per-parser transformations of data.
	444	The only way to make use of `pre_load <webargs.core.Parser.pre_load>` is to
	445	subclass a :class:`Parser <webargs.core.Parser>` and provide an
	446	implementation.
	447
	448	`pre_load <webargs.core.Parser.pre_load>` is given the data fetched from a
	449	location, the schema which will be used, the request object, and the location
	450	name which was requested. For example, to define a ``FlaskParser`` which strips
	451	whitespace from ``form`` and ``query`` data, one could write the following:
	452
	453	.. code-block:: python
	454
	455	from webargs.flaskparser import FlaskParser
	456	import typing
	457
	458
	459	def _strip_whitespace(value):
	460	if isinstance(value, str):
	461	value = value.strip()
	462	elif isinstance(value, typing.Mapping):
	463	return {k: _strip_whitespace(value[k]) for k in value}
	464	elif isinstance(value, (list, tuple)):
	465	return type(value)(map(_strip_whitespace, value))
	466	return value
	467
	468
	469	class WhitspaceStrippingFlaskParser(FlaskParser):
	470	def pre_load(self, location_data, *, schema, req, location):
	471	if location in ("query", "form"):
	472	return _strip_whitespace(location_data)
	473	return location_data
	474
	475	Note that `Parser.pre_load <webargs.core.Parser.pre_load>` is run after location
	476	loading but before ``Schema.load`` is called. It can therefore be called on
	477	multiple types of mapping objects, including
	478	:class:`MultiDictProxy <webargs.MultiDictProxy>`, depending on what the
	479	location loader returns.
	480
437	481	Returning HTTP 400 Responses
438	482	----------------------------
439	483

492	536	"""
493	537	# ...
494	538
	539	Multi-Field Detection
	540	---------------------
	541
	542	If a ``List`` field is used to parse data from a location like query parameters --
	543	where one or multiple values can be passed for a single parameter name -- then
	544	webargs will automatically treat that field as a list and parse multiple values
	545	if present.
	546
	547	To implement this behavior, webargs will examine schemas for ``marshmallow.fields.List``
	548	fields. ``List`` fields get unpacked to list values when data is loaded, and
	549	other fields do not. This also applies to fields which inherit from ``List``.
	550
	551	.. note::
	552
	553	In webargs v8, ``Tuple`` will be treated this way as well, in addition to ``List``.
	554
	555	What if you have a list which should be treated as a "multi-field" but which
	556	does not inherit from ``List``? webargs offers two solutions.
	557	You can add the custom attribute `is_multiple=True` to your field or you
	558	can add your class to your parser's list of `KNOWN_MULTI_FIELDS`.
	559
	560	First, let's define a "multiplexing field" which takes a string or list of
	561	strings to serve as an example:
	562
	563	.. code-block:: python
	564
	565	# a custom field class which can accept values like List(String()) or String()
	566	class CustomMultiplexingField(fields.String):
	567	def _deserialize(self, value, attr, data, **kwargs):
	568	if isinstance(value, str):
	569	return super()._deserialize(value, attr, data, **kwargs)
	570	return [
	571	self._deserialize(v, attr, data, **kwargs)
	572	for v in value
	573	if isinstance(v, str)
	574	]
	575
	576	def _serialize(self, value, attr, **kwargs):
	577	if isinstance(value, str):
	578	return super()._serialize(value, attr, **kwargs)
	579	return [self._serialize(v, attr, **kwargs) for v in value if isinstance(v, str)]
	580
	581
	582	If you control the definition of ``CustomMultiplexingField``, you can just add
	583	``is_multiple=True`` to it:
	584
	585	.. code-block:: python
	586
	587	# option 1: define the field with is_multiple = True
	588	from webargs.flaskparser import parser
	589
	590
	591	class CustomMultiplexingField(fields.Field):
	592	is_multiple = True # <----- this marks this as a multi-field
	593
	594	... # as above
	595
	596	If you don't control the definition of ``CustomMultiplexingField``, for example
	597	because it comes from a library, you can add it to the list of known
	598	multifields:
	599
	600	.. code-block:: python
	601
	602	# option 2: add the field to the parer's list of multi-fields
	603	class MyParser(FlaskParser):
	604	KNOWN_MULTI_FIELDS = list(FlaskParser.KNOWN_MULTI_FIELDS) + [
	605	CustomMultiplexingField
	606	]
	607
	608
	609	parser = MyParser()
	610
	611	In either case, the end result is that you can use the multifield and it will
	612	be detected as a list when unpacking query string data:
	613
	614	.. code-block:: python
	615
	616	# gracefully handles
	617	# ...?foo=a
	618	# ...?foo=a&foo=b
	619	# and treats them as ["a"] and ["a", "b"] respectively
	620	@parser.use_args({"foo": CustomMultiplexingField()}, location="query")
	621	def show_foos(foo):
	622	...
	623
	624
495	625	Mixing Locations
496	626	----------------
497	627

+4

-4

setup.py less more

19	19	]
20	20	+ FRAMEWORKS,
21	21	"lint": [
22		"mypy==0.790",
23		"flake8==3.8.4",
24		"flake8-bugbear==20.11.1",
	22	"mypy==0.812",
	23	"flake8==3.9.0",
	24	"flake8-bugbear==21.4.3",
25	25	"pre-commit~=2.4",
26	26	],
27		"docs": ["Sphinx==3.3.1", "sphinx-issues==1.2.0", "sphinx-typlog-theme==0.8.0"]
	27	"docs": ["Sphinx==3.5.3", "sphinx-issues==1.2.0", "sphinx-typlog-theme==0.8.0"]
28	28	+ FRAMEWORKS,
29	29	}
30	30	EXTRAS_REQUIRE["dev"] = EXTRAS_REQUIRE["tests"] + EXTRAS_REQUIRE["lint"] + ["tox"]

+1

-1

src/webargs/__init__.py less more

6	6	from webargs.core import ValidationError
7	7	from webargs import fields
8	8
9		__version__ = "7.0.1"
	9	__version__ = "8.0.0"
10	10	__version_info__ = tuple(LooseVersion(__version__).version)
11	11	__all__ = ("ValidationError", "fields", "missing", "validate")

+5

-5

src/webargs/aiohttpparser.py less more

70	70	class AIOHTTPParser(AsyncParser):
71	71	"""aiohttp request argument parser."""
72	72
73		DEFAULT_UNKNOWN_BY_LOCATION = {
	73	DEFAULT_UNKNOWN_BY_LOCATION: typing.Dict[str, typing.Optional[str]] = {
74	74	"match_info": RAISE,
75	75	"path": RAISE,
76	76	**core.Parser.DEFAULT_UNKNOWN_BY_LOCATION,

83	83
84	84	def load_querystring(self, req, schema: Schema) -> MultiDictProxy:
85	85	"""Return query params from the request as a MultiDictProxy."""
86		return MultiDictProxy(req.query, schema)
	86	return self._makeproxy(req.query, schema)
87	87
88	88	async def load_form(self, req, schema: Schema) -> MultiDictProxy:
89	89	"""Return form values from the request as a MultiDictProxy."""
90	90	post_data = await req.post()
91		return MultiDictProxy(post_data, schema)
	91	return self._makeproxy(post_data, schema)
92	92
93	93	async def load_json_or_form(
94	94	self, req, schema: Schema

113	113
114	114	def load_headers(self, req, schema: Schema) -> MultiDictProxy:
115	115	"""Return headers from the request as a MultiDictProxy."""
116		return MultiDictProxy(req.headers, schema)
	116	return self._makeproxy(req.headers, schema)
117	117
118	118	def load_cookies(self, req, schema: Schema) -> MultiDictProxy:
119	119	"""Return cookies from the request as a MultiDictProxy."""
120		return MultiDictProxy(req.cookies, schema)
	120	return self._makeproxy(req.cookies, schema)
121	121
122	122	def load_files(self, req, schema: Schema) -> typing.NoReturn:
123	123	raise NotImplementedError(

+4

-5

src/webargs/bottleparser.py less more

18	18	import bottle
19	19
20	20	from webargs import core
21		from webargs.multidictproxy import MultiDictProxy
22	21
23	22
24	23	class BottleParser(core.Parser):

48	47
49	48	def load_querystring(self, req, schema):
50	49	"""Return query params from the request as a MultiDictProxy."""
51		return MultiDictProxy(req.query, schema)
	50	return self._makeproxy(req.query, schema)
52	51
53	52	def load_form(self, req, schema):
54	53	"""Return form values from the request as a MultiDictProxy."""

57	56	# TODO: Make this check more specific
58	57	if core.is_json(req.content_type):
59	58	return core.missing
60		return MultiDictProxy(req.forms, schema)
	59	return self._makeproxy(req.forms, schema)
61	60
62	61	def load_headers(self, req, schema):
63	62	"""Return headers from the request as a MultiDictProxy."""
64		return MultiDictProxy(req.headers, schema)
	63	return self._makeproxy(req.headers, schema)
65	64
66	65	def load_cookies(self, req, schema):
67	66	"""Return cookies from the request."""

69	68
70	69	def load_files(self, req, schema):
71	70	"""Return files from the request as a MultiDictProxy."""
72		return MultiDictProxy(req.files, schema)
	71	return self._makeproxy(req.files, schema)
73	72
74	73	def handle_error(self, error, req, schema, *, error_status_code, error_headers):
75	74	"""Handles errors during parsing. Aborts the current request with a

+26

-12

src/webargs/core.py less more

7	7	from marshmallow import ValidationError
8	8	from marshmallow.utils import missing
9	9
10		from webargs.fields import DelimitedList
	10	from webargs.multidictproxy import MultiDictProxy
11	11
12	12	logger = logging.getLogger(__name__)
13	13
14	14
15	15	__all__ = [
16	16	"ValidationError",
17		"is_multiple",
18	17	"Parser",
19	18	"missing",
20	19	"parse_json",

54	53	if obj and not _iscallable(obj):
55	54	raise ValueError(f"{obj!r} is not callable.")
56	55	return obj
57
58
59		def is_multiple(field: ma.fields.Field) -> bool:
60		"""Return whether or not `field` handles repeated/multi-value arguments."""
61		return isinstance(field, ma.fields.List) and not isinstance(field, DelimitedList)
62	56
63	57
64	58	def get_mimetype(content_type: str) -> str:

131	125	DEFAULT_LOCATION: str = "json"
132	126	#: Default value to use for 'unknown' on schema load
133	127	# on a per-location basis
134		DEFAULT_UNKNOWN_BY_LOCATION: typing.Dict[str, str] = {
135		"json": ma.RAISE,
136		"form": ma.RAISE,
137		"json_or_form": ma.RAISE,
	128	DEFAULT_UNKNOWN_BY_LOCATION: typing.Dict[str, typing.Optional[str]] = {
	129	"json": None,
	130	"form": None,
	131	"json_or_form": None,
138	132	"querystring": ma.EXCLUDE,
139	133	"query": ma.EXCLUDE,
140	134	"headers": ma.EXCLUDE,

147	141	DEFAULT_VALIDATION_STATUS: int = DEFAULT_VALIDATION_STATUS
148	142	#: Default error message for validation errors
149	143	DEFAULT_VALIDATION_MESSAGE: str = "Invalid value."
	144	#: field types which should always be treated as if they set `is_multiple=True`
	145	KNOWN_MULTI_FIELDS: typing.List[typing.Type] = [ma.fields.List, ma.fields.Tuple]
150	146
151	147	#: Maps location => method name
152	148	__location_map__: typing.Dict[str, typing.Union[str, typing.Callable]] = {

174	170	)
175	171	self.schema_class = schema_class or self.DEFAULT_SCHEMA_CLASS
176	172	self.unknown = unknown
	173
	174	def _makeproxy(
	175	self, multidict, schema: ma.Schema, cls: typing.Type = MultiDictProxy
	176	):
	177	"""Create a multidict proxy object with options from the current parser"""
	178	return cls(multidict, schema, known_multi_fields=tuple(self.KNOWN_MULTI_FIELDS))
177	179
178	180	def _get_loader(self, location: str) -> typing.Callable:
179	181	"""Get the loader function for the given location.

319	321	location_data = self._load_location_data(
320	322	schema=schema, req=req, location=location
321	323	)
322		data = schema.load(location_data, **load_kwargs)
	324	preprocessed_data = self.pre_load(
	325	location_data, schema=schema, req=req, location=location
	326	)
	327	data = schema.load(preprocessed_data, **load_kwargs)
323	328	self._validate_arguments(data, validators)
324	329	except ma.exceptions.ValidationError as error:
325	330	self._on_validation_error(

520	525	self.error_callback = func
521	526	return func
522	527
	528	def pre_load(
	529	self, location_data: Mapping, *, schema: ma.Schema, req: Request, location: str
	530	) -> Mapping:
	531	"""A method of the parser which can transform data after location
	532	loading is done. By default it does nothing, but users can subclass
	533	parsers and override this method.
	534	"""
	535	return location_data
	536
523	537	def _handle_invalid_json_error(
524	538	self,
525	539	error: typing.Union[json.JSONDecodeError, UnicodeDecodeError],

+3

-4

src/webargs/djangoparser.py less more

17	17	return HttpResponse('Hello ' + args['name'])
18	18	"""
19	19	from webargs import core
20		from webargs.multidictproxy import MultiDictProxy
21	20
22	21
23	22	def is_json_request(req):

47	46
48	47	def load_querystring(self, req, schema):
49	48	"""Return query params from the request as a MultiDictProxy."""
50		return MultiDictProxy(req.GET, schema)
	49	return self._makeproxy(req.GET, schema)
51	50
52	51	def load_form(self, req, schema):
53	52	"""Return form values from the request as a MultiDictProxy."""
54		return MultiDictProxy(req.POST, schema)
	53	return self._makeproxy(req.POST, schema)
55	54
56	55	def load_cookies(self, req, schema):
57	56	"""Return cookies from the request."""

65	64
66	65	def load_files(self, req, schema):
67	66	"""Return files from the request as a MultiDictProxy."""
68		return MultiDictProxy(req.FILES, schema)
	67	return self._makeproxy(req.FILES, schema)
69	68
70	69	def get_request_from_view_args(self, view, args, kwargs):
71	70	# The first argument is either `self` or `request`

+2

-3

src/webargs/falconparser.py less more

5	5	import marshmallow as ma
6	6
7	7	from webargs import core
8		from webargs.multidictproxy import MultiDictProxy
9	8
10	9	HTTP_422 = "422 Unprocessable Entity"
11	10

96	95
97	96	def load_querystring(self, req, schema):
98	97	"""Return query params from the request as a MultiDictProxy."""
99		return MultiDictProxy(req.params, schema)
	98	return self._makeproxy(req.params, schema)
100	99
101	100	def load_form(self, req, schema):
102	101	"""Return form values from the request as a MultiDictProxy

108	107	form = parse_form_body(req)
109	108	if form is core.missing:
110	109	return form
111		return MultiDictProxy(form, schema)
	110	return self._makeproxy(form, schema)
112	111
113	112	def load_media(self, req, schema):
114	113	"""Return data unpacked and parsed by one of Falcon's media handlers.

+2

-0

src/webargs/fields.py less more

54	54	"""
55	55
56	56	delimiter: str = ","
	57	# delimited fields set is_multiple=False for webargs.core.is_multiple
	58	is_multiple: bool = False
57	59
58	60	def _serialize(self, value, attr, obj, **kwargs):
59	61	# serializing will start with parent-class serialization, so that we correctly

+7

-6

src/webargs/flaskparser.py less more

19	19	uid=uid, per_page=args["per_page"]
20	20	)
21	21	"""
	22	import typing
	23
22	24	import flask
23	25	from werkzeug.exceptions import HTTPException
24	26
25	27	import marshmallow as ma
26	28
27	29	from webargs import core
28		from webargs.multidictproxy import MultiDictProxy
29	30
30	31
31	32	def abort(http_status_code, exc=None, **kwargs):

49	50	class FlaskParser(core.Parser):
50	51	"""Flask request argument parser."""
51	52
52		DEFAULT_UNKNOWN_BY_LOCATION = {
	53	DEFAULT_UNKNOWN_BY_LOCATION: typing.Dict[str, typing.Optional[str]] = {
53	54	"view_args": ma.RAISE,
54	55	"path": ma.RAISE,
55	56	**core.Parser.DEFAULT_UNKNOWN_BY_LOCATION,

79	80
80	81	def load_querystring(self, req, schema):
81	82	"""Return query params from the request as a MultiDictProxy."""
82		return MultiDictProxy(req.args, schema)
	83	return self._makeproxy(req.args, schema)
83	84
84	85	def load_form(self, req, schema):
85	86	"""Return form values from the request as a MultiDictProxy."""
86		return MultiDictProxy(req.form, schema)
	87	return self._makeproxy(req.form, schema)
87	88
88	89	def load_headers(self, req, schema):
89	90	"""Return headers from the request as a MultiDictProxy."""
90		return MultiDictProxy(req.headers, schema)
	91	return self._makeproxy(req.headers, schema)
91	92
92	93	def load_cookies(self, req, schema):
93	94	"""Return cookies from the request."""

95	96
96	97	def load_files(self, req, schema):
97	98	"""Return files from the request as a MultiDictProxy."""
98		return MultiDictProxy(req.files, schema)
	99	return self._makeproxy(req.files, schema)
99	100
100	101	def handle_error(self, error, req, schema, *, error_status_code, error_headers):
101	102	"""Handles errors during parsing. Aborts the current HTTP request and

+24

-8

src/webargs/multidictproxy.py less more

0	0	from collections.abc import Mapping
	1	import typing
1	2
2	3	import marshmallow as ma
3
4		from webargs.core import missing, is_multiple
5	4
6	5
7	6	class MultiDictProxy(Mapping):

14	13	In all other cases, __getitem__ proxies directly to the input multidict.
15	14	"""
16	15
17		def __init__(self, multidict, schema: ma.Schema):
	16	def __init__(
	17	self,
	18	multidict,
	19	schema: ma.Schema,
	20	known_multi_fields: typing.Tuple[typing.Type, ...] = (
	21	ma.fields.List,
	22	ma.fields.Tuple,
	23	),
	24	):
18	25	self.data = multidict
	26	self.known_multi_fields = known_multi_fields
19	27	self.multiple_keys = self._collect_multiple_keys(schema)
20	28
21		@staticmethod
22		def _collect_multiple_keys(schema: ma.Schema):
	29	def _is_multiple(self, field: ma.fields.Field) -> bool:
	30	"""Return whether or not `field` handles repeated/multi-value arguments."""
	31	# fields which set `is_multiple = True/False` will have the value selected,
	32	# otherwise, we check for explicit criteria
	33	is_multiple_attr = getattr(field, "is_multiple", None)
	34	if is_multiple_attr is not None:
	35	return is_multiple_attr
	36	return isinstance(field, self.known_multi_fields)
	37
	38	def _collect_multiple_keys(self, schema: ma.Schema):
23	39	result = set()
24	40	for name, field in schema.fields.items():
25		if not is_multiple(field):
	41	if not self._is_multiple(field):
26	42	continue
27	43	result.add(field.data_key if field.data_key is not None else name)
28	44	return result
29	45
30	46	def __getitem__(self, key):
31		val = self.data.get(key, missing)
32		if val is missing or key not in self.multiple_keys:
	47	val = self.data.get(key, ma.missing)
	48	if val is ma.missing or key not in self.multiple_keys:
33	49	return val
34	50	if hasattr(self.data, "getlist"):
35	51	return self.data.getlist(key)

+8

-8

src/webargs/pyramidparser.py less more

24	24	server.serve_forever()
25	25	"""
26	26	import functools
	27	import typing
27	28	from collections.abc import Mapping
28	29
29	30	from webob.multidict import MultiDict

33	34
34	35	from webargs import core
35	36	from webargs.core import json
36		from webargs.multidictproxy import MultiDictProxy
37	37
38	38
39	39	def is_json_request(req):

43	43	class PyramidParser(core.Parser):
44	44	"""Pyramid request argument parser."""
45	45
46		DEFAULT_UNKNOWN_BY_LOCATION = {
	46	DEFAULT_UNKNOWN_BY_LOCATION: typing.Dict[str, typing.Optional[str]] = {
47	47	"matchdict": ma.RAISE,
48	48	"path": ma.RAISE,
49	49	**core.Parser.DEFAULT_UNKNOWN_BY_LOCATION,

66	66
67	67	def load_querystring(self, req, schema):
68	68	"""Return query params from the request as a MultiDictProxy."""
69		return MultiDictProxy(req.GET, schema)
	69	return self._makeproxy(req.GET, schema)
70	70
71	71	def load_form(self, req, schema):
72	72	"""Return form values from the request as a MultiDictProxy."""
73		return MultiDictProxy(req.POST, schema)
	73	return self._makeproxy(req.POST, schema)
74	74
75	75	def load_cookies(self, req, schema):
76	76	"""Return cookies from the request as a MultiDictProxy."""
77		return MultiDictProxy(req.cookies, schema)
	77	return self._makeproxy(req.cookies, schema)
78	78
79	79	def load_headers(self, req, schema):
80	80	"""Return headers from the request as a MultiDictProxy."""
81		return MultiDictProxy(req.headers, schema)
	81	return self._makeproxy(req.headers, schema)
82	82
83	83	def load_files(self, req, schema):
84	84	"""Return files from the request as a MultiDictProxy."""
85	85	files = ((k, v) for k, v in req.POST.items() if hasattr(v, "file"))
86		return MultiDictProxy(MultiDict(files), schema)
	86	return self._makeproxy(MultiDict(files), schema)
87	87
88	88	def load_matchdict(self, req, schema):
89	89	"""Return the request's ``matchdict`` as a MultiDictProxy."""
90		return MultiDictProxy(req.matchdict, schema)
	90	return self._makeproxy(req.matchdict, schema)
91	91
92	92	def handle_error(self, error, req, schema, *, error_status_code, error_headers):
93	93	"""Handles errors during parsing. Aborts the current HTTP request and

+11

-5

src/webargs/tornadoparser.py less more

96	96
97	97	def load_querystring(self, req, schema):
98	98	"""Return query params from the request as a MultiDictProxy."""
99		return WebArgsTornadoMultiDictProxy(req.query_arguments, schema)
	99	return self._makeproxy(
	100	req.query_arguments, schema, cls=WebArgsTornadoMultiDictProxy
	101	)
100	102
101	103	def load_form(self, req, schema):
102	104	"""Return form values from the request as a MultiDictProxy."""
103		return WebArgsTornadoMultiDictProxy(req.body_arguments, schema)
	105	return self._makeproxy(
	106	req.body_arguments, schema, cls=WebArgsTornadoMultiDictProxy
	107	)
104	108
105	109	def load_headers(self, req, schema):
106	110	"""Return headers from the request as a MultiDictProxy."""
107		return WebArgsTornadoMultiDictProxy(req.headers, schema)
	111	return self._makeproxy(req.headers, schema, cls=WebArgsTornadoMultiDictProxy)
108	112
109	113	def load_cookies(self, req, schema):
110	114	"""Return cookies from the request as a MultiDictProxy."""
111	115	# use the specialized subclass specifically for handling Tornado
112	116	# cookies
113		return WebArgsTornadoCookiesMultiDictProxy(req.cookies, schema)
	117	return self._makeproxy(
	118	req.cookies, schema, cls=WebArgsTornadoCookiesMultiDictProxy
	119	)
114	120
115	121	def load_files(self, req, schema):
116	122	"""Return files from the request as a MultiDictProxy."""
117		return WebArgsTornadoMultiDictProxy(req.files, schema)
	123	return self._makeproxy(req.files, schema, cls=WebArgsTornadoMultiDictProxy)
118	124
119	125	def handle_error(self, error, req, schema, *, error_status_code, error_headers):
120	126	"""Handles errors during parsing. Raises a `tornado.web.HTTPError`

+176

-1

tests/test_core.py less more

0	0	import datetime
	1	import typing
1	2	from unittest import mock
2	3
3	4	import pytest

34	35	"""A minimal parser implementation that parses mock requests."""
35	36
36	37	def load_querystring(self, req, schema):
37		return MultiDictProxy(req.query, schema)
	38	return self._makeproxy(req.query, schema)
	39
	40	def load_form(self, req, schema):
	41	return MultiDictProxy(req.form, schema)
38	42
39	43	def load_json(self, req, schema):
40	44	return req.json

1031	1035	parser.parse(args, web_request)
1032	1036
1033	1037
	1038	@pytest.mark.parametrize("input_dict", multidicts)
	1039	@pytest.mark.parametrize(
	1040	"setting",
	1041	[
	1042	"is_multiple_true",
	1043	"is_multiple_false",
	1044	"is_multiple_notset",
	1045	"list_field",
	1046	"tuple_field",
	1047	"added_to_known",
	1048	],
	1049	)
	1050	def test_is_multiple_detection(web_request, parser, input_dict, setting):
	1051	# this custom class "multiplexes" in that it can be given a single value or
	1052	# list of values -- a single value is treated as a string, and a list of
	1053	# values is treated as a list of strings
	1054	class CustomMultiplexingField(fields.String):
	1055	def _deserialize(self, value, attr, data, **kwargs):
	1056	if isinstance(value, str):
	1057	return super()._deserialize(value, attr, data, **kwargs)
	1058	return [
	1059	self._deserialize(v, attr, data, **kwargs)
	1060	for v in value
	1061	if isinstance(v, str)
	1062	]
	1063
	1064	def _serialize(self, value, attr, **kwargs):
	1065	if isinstance(value, str):
	1066	return super()._serialize(value, attr, **kwargs)
	1067	return [
	1068	self._serialize(v, attr, **kwargs) for v in value if isinstance(v, str)
	1069	]
	1070
	1071	class CustomMultipleField(CustomMultiplexingField):
	1072	is_multiple = True
	1073
	1074	class CustomNonMultipleField(CustomMultiplexingField):
	1075	is_multiple = False
	1076
	1077	# the request's query params are the input multidict
	1078	web_request.query = input_dict
	1079
	1080	# case 1: is_multiple=True
	1081	if setting == "is_multiple_true":
	1082	# the multidict should unpack to a list of strings
	1083	#
	1084	# order is not necessarily guaranteed by the multidict implementations, but
	1085	# both values must be present
	1086	args = {"foos": CustomMultipleField()}
	1087	result = parser.parse(args, web_request, location="query")
	1088	assert result["foos"] in (["a", "b"], ["b", "a"])
	1089	# case 2: is_multiple=False
	1090	elif setting == "is_multiple_false":
	1091	# the multidict should unpack to a string
	1092	#
	1093	# either value may be returned, depending on the multidict implementation,
	1094	# but not both
	1095	args = {"foos": CustomNonMultipleField()}
	1096	result = parser.parse(args, web_request, location="query")
	1097	assert result["foos"] in ("a", "b")
	1098	# case 3: is_multiple is not set
	1099	elif setting == "is_multiple_notset":
	1100	# this should be the same as is_multiple=False
	1101	args = {"foos": CustomMultiplexingField()}
	1102	result = parser.parse(args, web_request, location="query")
	1103	assert result["foos"] in ("a", "b")
	1104	# case 4: the field is a List (special case)
	1105	elif setting == "list_field":
	1106	# this should behave like the is_multiple=True case
	1107	args = {"foos": fields.List(fields.Str())}
	1108	result = parser.parse(args, web_request, location="query")
	1109	assert result["foos"] in (["a", "b"], ["b", "a"])
	1110	# case 5: the field is a Tuple (special case)
	1111	elif setting == "tuple_field":
	1112	# this should behave like the is_multiple=True case and produce a tuple
	1113	args = {"foos": fields.Tuple((fields.Str, fields.Str))}
	1114	result = parser.parse(args, web_request, location="query")
	1115	assert result["foos"] in (("a", "b"), ("b", "a"))
	1116	# case 6: the field is custom, but added to the known fields of the parser
	1117	elif setting == "added_to_known":
	1118	# if it's included in the known multifields and is_multiple is not set, behave
	1119	# like is_multiple=True
	1120	parser.KNOWN_MULTI_FIELDS.append(CustomMultiplexingField)
	1121	args = {"foos": CustomMultiplexingField()}
	1122	result = parser.parse(args, web_request, location="query")
	1123	assert result["foos"] in (["a", "b"], ["b", "a"])
	1124	else:
	1125	raise NotImplementedError
	1126
	1127
1034	1128	def test_validation_errors_in_validator_are_passed_to_handle_error(parser, web_request):
1035	1129	def validate(value):
1036	1130	raise ValidationError("Something went wrong.")

1133	1227	p = CustomParser()
1134	1228	ret = p.parse(argmap, web_request)
1135	1229	assert ret == {"value": "hello world"}
	1230
	1231
	1232	def test_parser_pre_load(web_request):
	1233	class CustomParser(MockRequestParser):
	1234	# pre-load hook to strip whitespace from query params
	1235	def pre_load(self, data, *, schema, req, location):
	1236	if location == "query":
	1237	return {k: v.strip() for k, v in data.items()}
	1238	return data
	1239
	1240	parser = CustomParser()
	1241
	1242	# mock data for both query and json
	1243	web_request.query = web_request.json = {"value": " hello "}
	1244	argmap = {"value": fields.Str()}
	1245
	1246	# data gets through for 'json' just fine
	1247	ret = parser.parse(argmap, web_request)
	1248	assert ret == {"value": " hello "}
	1249
	1250	# but for 'query', the pre_load hook changes things
	1251	ret = parser.parse(argmap, web_request, location="query")
	1252	assert ret == {"value": "hello"}
	1253
	1254
	1255	# this test is meant to be a run of the WhitspaceStrippingFlaskParser we give
	1256	# in the docs/advanced.rst examples for how to use pre_load
	1257	# this helps ensure that the example code is correct
	1258	# rather than a FlaskParser, we're working with the mock parser, but it's
	1259	# otherwise the same
	1260	def test_whitespace_stripping_parser_example(web_request):
	1261	def _strip_whitespace(value):
	1262	if isinstance(value, str):
	1263	value = value.strip()
	1264	elif isinstance(value, typing.Mapping):
	1265	return {k: _strip_whitespace(value[k]) for k in value}
	1266	elif isinstance(value, (list, tuple)):
	1267	return type(value)(map(_strip_whitespace, value))
	1268	return value
	1269
	1270	class WhitspaceStrippingParser(MockRequestParser):
	1271	def pre_load(self, location_data, *, schema, req, location):
	1272	if location in ("query", "form"):
	1273	ret = _strip_whitespace(location_data)
	1274	return ret
	1275	return location_data
	1276
	1277	parser = WhitspaceStrippingParser()
	1278
	1279	# mock data for query, form, and json
	1280	web_request.form = web_request.query = web_request.json = {"value": " hello "}
	1281	argmap = {"value": fields.Str()}
	1282
	1283	# data gets through for 'json' just fine
	1284	ret = parser.parse(argmap, web_request)
	1285	assert ret == {"value": " hello "}
	1286
	1287	# but for 'query' and 'form', the pre_load hook changes things
	1288	for loc in ("query", "form"):
	1289	ret = parser.parse(argmap, web_request, location=loc)
	1290	assert ret == {"value": "hello"}
	1291
	1292	# check that it applies in the case where the field is a list type
	1293	# applied to an argument (logic for `tuple` is effectively the same)
	1294	web_request.form = web_request.query = web_request.json = {
	1295	"ids": [" 1", "3", " 4"],
	1296	"values": [" foo ", " bar"],
	1297	}
	1298	schema = Schema.from_dict(
	1299	{"ids": fields.List(fields.Int), "values": fields.List(fields.Str)}
	1300	)
	1301	for loc in ("query", "form"):
	1302	ret = parser.parse(schema, web_request, location=loc)
	1303	assert ret == {"ids": [1, 3, 4], "values": ["foo", "bar"]}
	1304
	1305	# json loading should also work even though the pre_load hook above
	1306	# doesn't strip whitespace from JSON data
	1307	# - values=[" foo ", ...] will have whitespace preserved
	1308	# - ids=[" 1", ...] will still parse okay because " 1" is valid for fields.Int
	1309	ret = parser.parse(schema, web_request, location="json")
	1310	assert ret == {"ids": [1, 3, 4], "values": [" foo ", " bar"]}

+1

-0

tox.ini less more

29	29	# issues in which `mypy` running on every file standalone won't catch things
30	30	[testenv:mypy]
31	31	deps = mypy
	32	extras = frameworks
32	33	commands = mypy src/
33	34
34	35	[testenv:docs]